ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning Article Swipe
Vamsi Aribandi
,
Yi Tay
,
Tal Schuster
,
Jinfeng Rao
,
Huaixiu Zheng
,
Sanket Vaibhav Mehta
,
Honglei Zhuang
,
Vinh Q. Tran
,
Dara Bahri
,
Jianmo Ni
,
Jai Prakash Gupta
,
Kai Hui
,
Sebastian Ruder
,
Donald Metzler
·
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2111.10952
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2111.10952
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the number of tasks during pre-training. Towards this goal, this paper introduces ExMix (Extreme Mixture): a massive collection of 107 supervised NLP tasks across diverse domains and task-families. Using ExMix, we study the effect of multi-task pre-training at the largest scale to date, and analyze co-training transfer amongst common families of tasks. Through this analysis, we show that manually curating an ideal set of tasks for multi-task pre-training is not straightforward, and that multi-task scaling can vastly improve models on its own. Finally, we propose ExT5: a model pre-trained using a multi-task objective of self-supervised span denoising and supervised ExMix. Via extensive experiments, we show that ExT5 outperforms strong T5 baselines on SuperGLUE, GEM, Rainbow, Closed-Book QA tasks, and several tasks outside of ExMix. ExT5 also significantly improves sample efficiency while pre-training.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2111.10952
- https://arxiv.org/pdf/2111.10952
- OA Status
- green
- Cited By
- 8
- References
- 126
- OpenAlex ID
- https://openalex.org/W3217187998
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W3217187998Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2111.10952Digital Object Identifier
- Title
-
ExT5: Towards Extreme Multi-Task Scaling for Transfer LearningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2021Year of publication
- Publication date
-
2021-11-22Full publication date if available
- Authors
-
Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q. Tran, Dara Bahri, Jianmo Ni, Jai Prakash Gupta, Kai Hui, Sebastian Ruder, Donald MetzlerList of authors in order
- Landing page
-
https://arxiv.org/abs/2111.10952Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2111.10952Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2111.10952Direct OA link when available
- Concepts
-
Computer science, Task (project management), Transfer of learning, Artificial intelligence, Machine learning, Set (abstract data type), Scaling, Natural language processing, Multi-task learning, Mathematics, Programming language, Geometry, Management, EconomicsTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
8Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1, 2023: 4, 2022: 2, 2021: 1Per-year citation counts (last 5 years)
- References (count)
-
126Number of works referenced by this work
Full payload
| id | https://openalex.org/W3217187998 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2111.10952 |
| ids.doi | https://doi.org/10.48550/arxiv.2111.10952 |
| ids.mag | 3217187998 |
| ids.openalex | https://openalex.org/W3217187998 |
| fwci | |
| type | preprint |
| title | ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10028 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 1.0 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Topic Modeling |
| topics[1].id | https://openalex.org/T10181 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9993000030517578 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Natural Language Processing Techniques |
| topics[2].id | https://openalex.org/T11307 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9991999864578247 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Domain Adaptation and Few-Shot Learning |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C41008148 |
| concepts[0].level | 0 |
| concepts[0].score | 0.7943284511566162 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[0].display_name | Computer science |
| concepts[1].id | https://openalex.org/C2780451532 |
| concepts[1].level | 2 |
| concepts[1].score | 0.7592452764511108 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[1].display_name | Task (project management) |
| concepts[2].id | https://openalex.org/C150899416 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6856981515884399 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q1820378 |
| concepts[2].display_name | Transfer of learning |
| concepts[3].id | https://openalex.org/C154945302 |
| concepts[3].level | 1 |
| concepts[3].score | 0.5924543142318726 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[3].display_name | Artificial intelligence |
| concepts[4].id | https://openalex.org/C119857082 |
| concepts[4].level | 1 |
| concepts[4].score | 0.5562905669212341 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q2539 |
| concepts[4].display_name | Machine learning |
| concepts[5].id | https://openalex.org/C177264268 |
| concepts[5].level | 2 |
| concepts[5].score | 0.5394900441169739 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1514741 |
| concepts[5].display_name | Set (abstract data type) |
| concepts[6].id | https://openalex.org/C99844830 |
| concepts[6].level | 2 |
| concepts[6].score | 0.43391168117523193 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q102441924 |
| concepts[6].display_name | Scaling |
| concepts[7].id | https://openalex.org/C204321447 |
| concepts[7].level | 1 |
| concepts[7].score | 0.4315984547138214 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[7].display_name | Natural language processing |
| concepts[8].id | https://openalex.org/C28006648 |
| concepts[8].level | 3 |
| concepts[8].score | 0.4133259057998657 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q6934509 |
| concepts[8].display_name | Multi-task learning |
| concepts[9].id | https://openalex.org/C33923547 |
| concepts[9].level | 0 |
| concepts[9].score | 0.06835860013961792 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q395 |
| concepts[9].display_name | Mathematics |
| concepts[10].id | https://openalex.org/C199360897 |
| concepts[10].level | 1 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[10].display_name | Programming language |
| concepts[11].id | https://openalex.org/C2524010 |
| concepts[11].level | 1 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q8087 |
| concepts[11].display_name | Geometry |
| concepts[12].id | https://openalex.org/C187736073 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q2920921 |
| concepts[12].display_name | Management |
| concepts[13].id | https://openalex.org/C162324750 |
| concepts[13].level | 0 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[13].display_name | Economics |
| keywords[0].id | https://openalex.org/keywords/computer-science |
| keywords[0].score | 0.7943284511566162 |
| keywords[0].display_name | Computer science |
| keywords[1].id | https://openalex.org/keywords/task |
| keywords[1].score | 0.7592452764511108 |
| keywords[1].display_name | Task (project management) |
| keywords[2].id | https://openalex.org/keywords/transfer-of-learning |
| keywords[2].score | 0.6856981515884399 |
| keywords[2].display_name | Transfer of learning |
| keywords[3].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[3].score | 0.5924543142318726 |
| keywords[3].display_name | Artificial intelligence |
| keywords[4].id | https://openalex.org/keywords/machine-learning |
| keywords[4].score | 0.5562905669212341 |
| keywords[4].display_name | Machine learning |
| keywords[5].id | https://openalex.org/keywords/set |
| keywords[5].score | 0.5394900441169739 |
| keywords[5].display_name | Set (abstract data type) |
| keywords[6].id | https://openalex.org/keywords/scaling |
| keywords[6].score | 0.43391168117523193 |
| keywords[6].display_name | Scaling |
| keywords[7].id | https://openalex.org/keywords/natural-language-processing |
| keywords[7].score | 0.4315984547138214 |
| keywords[7].display_name | Natural language processing |
| keywords[8].id | https://openalex.org/keywords/multi-task-learning |
| keywords[8].score | 0.4133259057998657 |
| keywords[8].display_name | Multi-task learning |
| keywords[9].id | https://openalex.org/keywords/mathematics |
| keywords[9].score | 0.06835860013961792 |
| keywords[9].display_name | Mathematics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2111.10952 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2111.10952 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2111.10952 |
| locations[1].id | mag:3217187998 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | submittedVersion |
| locations[1].raw_type | |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | False |
| locations[1].raw_source_name | arXiv (Cornell University) |
| locations[1].landing_page_url | http://arxiv.org/pdf/2111.10952.pdf |
| locations[2].id | doi:10.48550/arxiv.2111.10952 |
| locations[2].is_oa | True |
| locations[2].source.id | https://openalex.org/S4306400194 |
| locations[2].source.issn | |
| locations[2].source.type | repository |
| locations[2].source.is_oa | True |
| locations[2].source.issn_l | |
| locations[2].source.is_core | False |
| locations[2].source.is_in_doaj | False |
| locations[2].source.display_name | arXiv (Cornell University) |
| locations[2].source.host_organization | https://openalex.org/I205783295 |
| locations[2].source.host_organization_name | Cornell University |
| locations[2].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[2].license | cc-by |
| locations[2].pdf_url | |
| locations[2].version | |
| locations[2].raw_type | article |
| locations[2].license_id | https://openalex.org/licenses/cc-by |
| locations[2].is_accepted | False |
| locations[2].is_published | |
| locations[2].raw_source_name | |
| locations[2].landing_page_url | https://doi.org/10.48550/arxiv.2111.10952 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5058345225 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-4345-1763 |
| authorships[0].author.display_name | Vamsi Aribandi |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Vamsi Aribandi |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5103069680 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-6896-4496 |
| authorships[1].author.display_name | Yi Tay |
| authorships[1].countries | US |
| authorships[1].affiliations[0].institution_ids | https://openalex.org/I1291425158 |
| authorships[1].affiliations[0].raw_affiliation_string | Google,,,,, |
| authorships[1].institutions[0].id | https://openalex.org/I1291425158 |
| authorships[1].institutions[0].ror | https://ror.org/00njsd438 |
| authorships[1].institutions[0].type | company |
| authorships[1].institutions[0].lineage | https://openalex.org/I1291425158, https://openalex.org/I4210128969 |
| authorships[1].institutions[0].country_code | US |
| authorships[1].institutions[0].display_name | Google (United States) |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Yi Tay |
| authorships[1].is_corresponding | False |
| authorships[1].raw_affiliation_strings | Google,,,,, |
| authorships[2].author.id | https://openalex.org/A5058037733 |
| authorships[2].author.orcid | https://orcid.org/0000-0002-7772-8230 |
| authorships[2].author.display_name | Tal Schuster |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Tal Schuster |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5075728689 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Jinfeng Rao |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Jinfeng Rao |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5003807260 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Huaixiu Zheng |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Huaixiu Steven Zheng |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5073777588 |
| authorships[5].author.orcid | |
| authorships[5].author.display_name | Sanket Vaibhav Mehta |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Sanket Vaibhav Mehta |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5011279860 |
| authorships[6].author.orcid | https://orcid.org/0000-0001-8134-1509 |
| authorships[6].author.display_name | Honglei Zhuang |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Honglei Zhuang |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5102435043 |
| authorships[7].author.orcid | |
| authorships[7].author.display_name | Vinh Q. Tran |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Vinh Q. Tran |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5036477705 |
| authorships[8].author.orcid | https://orcid.org/0000-0003-0144-2911 |
| authorships[8].author.display_name | Dara Bahri |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Dara Bahri |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5077817759 |
| authorships[9].author.orcid | https://orcid.org/0000-0002-6863-8073 |
| authorships[9].author.display_name | Jianmo Ni |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Jianmo Ni |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5008444234 |
| authorships[10].author.orcid | https://orcid.org/0009-0009-3233-7170 |
| authorships[10].author.display_name | Jai Prakash Gupta |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Jai Prakash Gupta |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5017775632 |
| authorships[11].author.orcid | https://orcid.org/0000-0002-3110-7404 |
| authorships[11].author.display_name | Kai Hui |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Kai Hui |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5037310413 |
| authorships[12].author.orcid | |
| authorships[12].author.display_name | Sebastian Ruder |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Sebastian Ruder |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5000115067 |
| authorships[13].author.orcid | https://orcid.org/0000-0003-4276-6269 |
| authorships[13].author.display_name | Donald Metzler |
| authorships[13].author_position | last |
| authorships[13].raw_author_name | Donald Metzler |
| authorships[13].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2111.10952 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10028 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 1.0 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Topic Modeling |
| cited_by_count | 8 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 4 |
| counts_by_year[2].year | 2022 |
| counts_by_year[2].cited_by_count | 2 |
| counts_by_year[3].year | 2021 |
| counts_by_year[3].cited_by_count | 1 |
| locations_count | 3 |
| best_oa_location.id | pmh:oai:arXiv.org:2111.10952 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2111.10952 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2111.10952 |
| primary_location.id | pmh:oai:arXiv.org:2111.10952 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2111.10952 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2111.10952 |
| publication_date | 2021-11-22 |
| publication_year | 2021 |
| referenced_works | https://openalex.org/W3100107515, https://openalex.org/W2997359900, https://openalex.org/W2888482885, https://openalex.org/W2807333695, https://openalex.org/W2252136820, https://openalex.org/W1599016936, https://openalex.org/W3006439205, https://openalex.org/W2460159515, https://openalex.org/W3169754167, https://openalex.org/W2145755360, https://openalex.org/W3213750647, https://openalex.org/W2606964149, https://openalex.org/W2806198715, https://openalex.org/W3199241049, https://openalex.org/W2525127255, https://openalex.org/W2998617917, https://openalex.org/W167809298, https://openalex.org/W3198002980, https://openalex.org/W2950733326, https://openalex.org/W3026404337, https://openalex.org/W3169283738, https://openalex.org/W3035199167, https://openalex.org/W3104739822, https://openalex.org/W3102187933, https://openalex.org/W2970780738, https://openalex.org/W2963341956, https://openalex.org/W2606974598, https://openalex.org/W3083978629, https://openalex.org/W2953271402, https://openalex.org/W2950681488, https://openalex.org/W2040916592, https://openalex.org/W2953958347, https://openalex.org/W2988092105, https://openalex.org/W2963854351, https://openalex.org/W2996176596, https://openalex.org/W2963168538, https://openalex.org/W2963846996, https://openalex.org/W2982399380, https://openalex.org/W3035008906, https://openalex.org/W3082274269, https://openalex.org/W3168429709, https://openalex.org/W3200043857, https://openalex.org/W2970062726, https://openalex.org/W54398672, https://openalex.org/W3098068947, https://openalex.org/W3011411500, https://openalex.org/W2888302696, https://openalex.org/W2963339397, https://openalex.org/W2739874095, https://openalex.org/W2913340405, https://openalex.org/W3034323190, https://openalex.org/W3154669786, https://openalex.org/W3034850762, https://openalex.org/W3045492832, https://openalex.org/W2956105246, https://openalex.org/W2144578941, https://openalex.org/W3035503910, https://openalex.org/W3186655327, https://openalex.org/W2996064239, https://openalex.org/W3106445907, https://openalex.org/W2963241138, https://openalex.org/W3124687886, https://openalex.org/W3099299360, https://openalex.org/W2916132663, https://openalex.org/W2912924812, https://openalex.org/W3106356412, https://openalex.org/W2251939518, https://openalex.org/W2534253848, https://openalex.org/W2606854773, https://openalex.org/W2946609015, https://openalex.org/W2804897457, https://openalex.org/W2951181836, https://openalex.org/W2963310665, https://openalex.org/W2978670439, https://openalex.org/W3170180819, https://openalex.org/W3169934659, https://openalex.org/W2113459411, https://openalex.org/W2954226438, https://openalex.org/W2990704537, https://openalex.org/W3102659883, https://openalex.org/W2966087730, https://openalex.org/W2117130368, https://openalex.org/W2257408573, https://openalex.org/W3123161422, https://openalex.org/W3039127676, https://openalex.org/W2892248135, https://openalex.org/W3001279689, https://openalex.org/W3172942063, https://openalex.org/W2963702144, https://openalex.org/W2962965405, https://openalex.org/W2898700502, https://openalex.org/W2251994258, https://openalex.org/W3194676777, https://openalex.org/W3176119108, https://openalex.org/W3022814719, https://openalex.org/W2540646130, https://openalex.org/W3035579820, https://openalex.org/W3104033643, https://openalex.org/W1970409510, https://openalex.org/W2963475460, https://openalex.org/W2964121793, https://openalex.org/W3133702157, https://openalex.org/W2920807444, https://openalex.org/W3035219538, https://openalex.org/W3099215402, https://openalex.org/W3098903812, https://openalex.org/W2964024811, https://openalex.org/W2922580172, https://openalex.org/W2963809228, https://openalex.org/W2964125718, https://openalex.org/W3128413221, https://openalex.org/W3034238904, https://openalex.org/W2889787757, https://openalex.org/W2975059944, https://openalex.org/W3136149525, https://openalex.org/W3099655892, https://openalex.org/W2995643077, https://openalex.org/W2963748441, https://openalex.org/W2951534261, https://openalex.org/W3131933120, https://openalex.org/W3175060421, https://openalex.org/W131533222, https://openalex.org/W3104240813, https://openalex.org/W2970316683, https://openalex.org/W3152574567, https://openalex.org/W2903697572 |
| referenced_works_count | 126 |
| abstract_inverted_index.a | 40, 111, 115 |
| abstract_inverted_index.QA | 141 |
| abstract_inverted_index.T5 | 134 |
| abstract_inverted_index.an | 85 |
| abstract_inverted_index.at | 62 |
| abstract_inverted_index.is | 93 |
| abstract_inverted_index.of | 4, 22, 27, 43, 59, 75, 88, 118, 147 |
| abstract_inverted_index.on | 104, 136 |
| abstract_inverted_index.to | 66 |
| abstract_inverted_index.up | 24 |
| abstract_inverted_index.we | 55, 80, 108, 128 |
| abstract_inverted_index.107 | 44 |
| abstract_inverted_index.NLP | 46 |
| abstract_inverted_index.Via | 125 |
| abstract_inverted_index.and | 7, 51, 68, 96, 122, 143 |
| abstract_inverted_index.can | 100 |
| abstract_inverted_index.few | 15 |
| abstract_inverted_index.for | 10, 90 |
| abstract_inverted_index.its | 105 |
| abstract_inverted_index.not | 94 |
| abstract_inverted_index.set | 87 |
| abstract_inverted_index.the | 1, 20, 25, 57, 63 |
| abstract_inverted_index.ExT5 | 131, 149 |
| abstract_inverted_index.GEM, | 138 |
| abstract_inverted_index.also | 150 |
| abstract_inverted_index.have | 17 |
| abstract_inverted_index.own. | 106 |
| abstract_inverted_index.show | 81, 129 |
| abstract_inverted_index.span | 120 |
| abstract_inverted_index.that | 82, 97, 130 |
| abstract_inverted_index.this | 32, 34, 78 |
| abstract_inverted_index.ExMix | 37 |
| abstract_inverted_index.ExT5: | 110 |
| abstract_inverted_index.Using | 53 |
| abstract_inverted_index.date, | 67 |
| abstract_inverted_index.goal, | 33 |
| abstract_inverted_index.ideal | 86 |
| abstract_inverted_index.model | 112 |
| abstract_inverted_index.paper | 35 |
| abstract_inverted_index.scale | 65 |
| abstract_inverted_index.study | 56 |
| abstract_inverted_index.tasks | 28, 47, 89, 145 |
| abstract_inverted_index.using | 114 |
| abstract_inverted_index.while | 155 |
| abstract_inverted_index.works | 16 |
| abstract_inverted_index.(NLP), | 14 |
| abstract_inverted_index.ExMix, | 54 |
| abstract_inverted_index.ExMix. | 124, 148 |
| abstract_inverted_index.across | 48 |
| abstract_inverted_index.common | 73 |
| abstract_inverted_index.during | 29 |
| abstract_inverted_index.effect | 21, 58 |
| abstract_inverted_index.models | 103 |
| abstract_inverted_index.number | 26 |
| abstract_inverted_index.recent | 2 |
| abstract_inverted_index.sample | 153 |
| abstract_inverted_index.strong | 133 |
| abstract_inverted_index.tasks, | 142 |
| abstract_inverted_index.tasks. | 76 |
| abstract_inverted_index.vastly | 101 |
| abstract_inverted_index.Despite | 0 |
| abstract_inverted_index.Through | 77 |
| abstract_inverted_index.Towards | 31 |
| abstract_inverted_index.amongst | 72 |
| abstract_inverted_index.analyze | 69 |
| abstract_inverted_index.diverse | 49 |
| abstract_inverted_index.domains | 50 |
| abstract_inverted_index.improve | 102 |
| abstract_inverted_index.largest | 64 |
| abstract_inverted_index.massive | 41 |
| abstract_inverted_index.natural | 11 |
| abstract_inverted_index.outside | 146 |
| abstract_inverted_index.propose | 109 |
| abstract_inverted_index.scaling | 23, 99 |
| abstract_inverted_index.several | 144 |
| abstract_inverted_index.studied | 19 |
| abstract_inverted_index.success | 3 |
| abstract_inverted_index.(Extreme | 38 |
| abstract_inverted_index.Finally, | 107 |
| abstract_inverted_index.Rainbow, | 139 |
| abstract_inverted_index.curating | 84 |
| abstract_inverted_index.families | 74 |
| abstract_inverted_index.improves | 152 |
| abstract_inverted_index.language | 12 |
| abstract_inverted_index.learning | 6, 9 |
| abstract_inverted_index.manually | 83 |
| abstract_inverted_index.transfer | 8, 71 |
| abstract_inverted_index.Mixture): | 39 |
| abstract_inverted_index.analysis, | 79 |
| abstract_inverted_index.baselines | 135 |
| abstract_inverted_index.denoising | 121 |
| abstract_inverted_index.extensive | 126 |
| abstract_inverted_index.objective | 117 |
| abstract_inverted_index.SuperGLUE, | 137 |
| abstract_inverted_index.collection | 42 |
| abstract_inverted_index.efficiency | 154 |
| abstract_inverted_index.introduces | 36 |
| abstract_inverted_index.multi-task | 5, 60, 91, 98, 116 |
| abstract_inverted_index.processing | 13 |
| abstract_inverted_index.supervised | 45, 123 |
| abstract_inverted_index.Closed-Book | 140 |
| abstract_inverted_index.co-training | 70 |
| abstract_inverted_index.outperforms | 132 |
| abstract_inverted_index.pre-trained | 113 |
| abstract_inverted_index.experiments, | 127 |
| abstract_inverted_index.pre-training | 61, 92 |
| abstract_inverted_index.pre-training. | 30, 156 |
| abstract_inverted_index.significantly | 151 |
| abstract_inverted_index.systematically | 18 |
| abstract_inverted_index.task-families. | 52 |
| abstract_inverted_index.self-supervised | 119 |
| abstract_inverted_index.straightforward, | 95 |
| cited_by_percentile_year | |
| countries_distinct_count | 1 |
| institutions_distinct_count | 14 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.7900000214576721 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |