Ensembling of Distilled Models from Multi-task Teachers for Constrained\n Resource Language Pairs Article Swipe
YOU?
·
· 2021
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2111.13284
This paper describes our submission to the constrained track of WMT21 shared\nnews translation task. We focus on the three relatively low resource language\npairs Bengali to and from Hindi, English to and from Hausa, and Xhosa to and\nfrom Zulu. To overcome the limitation of relatively low parallel data we train\na multilingual model using a multitask objective employing both parallel and\nmonolingual data. In addition, we augment the data using back translation. We\nalso train a bilingual model incorporating back translation and knowledge\ndistillation then combine the two models using sequence-to-sequence mapping. We\nsee around 70% relative gain in BLEU point for English to and from Hausa, and\naround 25% relative improvements for both Bengali to and from Hindi, and Xhosa\nto and from Zulu compared to bilingual baselines.\n
Related Topics
- Type
- preprint
- Landing Page
- http://arxiv.org/abs/2111.13284
- https://arxiv.org/pdf/2111.13284
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4286850516
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4286850516Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2111.13284Digital Object Identifier
- Title
-
Ensembling of Distilled Models from Multi-task Teachers for Constrained\n Resource Language PairsWork title
- Type
-
preprintOpenAlex work type
- Publication year
-
2021Year of publication
- Publication date
-
2021-11-25Full publication date if available
- Authors
-
Amr Hendy, Esraa A. Gad, Mohamed Abdelghaffar, Jailan S. ElMosalami, Mohamed Afify, Ahmed Y. Tawfik, Hany Hassan AwadallaList of authors in order
- Landing page
-
https://arxiv.org/abs/2111.13284Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2111.13284Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2111.13284Direct OA link when available
- Concepts
-
Bengali, Computer science, Zulu, Task (project management), Natural language processing, Hindi, Artificial intelligence, Machine translation, Hausa, Xhosa, Focus (optics), WordNet, Contrast (vision), Resource (disambiguation), Linguistics, Computer network, Economics, Management, Physics, Optics, PhilosophyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4286850516 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2111.13284 |
| ids.openalex | https://openalex.org/W4286850516 |
| fwci | 0.0 |
| type | preprint |
| title | Ensembling of Distilled Models from Multi-task Teachers for Constrained\n Resource Language Pairs |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9990000128746033 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T10028 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9951000213623047 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Topic Modeling |
| topics[2].id | https://openalex.org/T11714 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9945999979972839 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1707 |
| topics[2].subfield.display_name | Computer Vision and Pattern Recognition |
| topics[2].display_name | Multimodal Machine Learning Applications |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C19235068 |
| concepts[0].level | 2 |
| concepts[0].score | 0.8861085772514343 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q9610 |
| concepts[0].display_name | Bengali |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.7849254608154297 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C2778105672 |
| concepts[2].level | 2 |
| concepts[2].score | 0.771683931350708 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q10179 |
| concepts[2].display_name | Zulu |
| concepts[3].id | https://openalex.org/C2780451532 |
| concepts[3].level | 2 |
| concepts[3].score | 0.6971642971038818 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q759676 |
| concepts[3].display_name | Task (project management) |
| concepts[4].id | https://openalex.org/C204321447 |
| concepts[4].level | 1 |
| concepts[4].score | 0.6872802376747131 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[4].display_name | Natural language processing |
| concepts[5].id | https://openalex.org/C519982507 |
| concepts[5].level | 2 |
| concepts[5].score | 0.6831809282302856 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1568 |
| concepts[5].display_name | Hindi |
| concepts[6].id | https://openalex.org/C154945302 |
| concepts[6].level | 1 |
| concepts[6].score | 0.6371247172355652 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[6].display_name | Artificial intelligence |
| concepts[7].id | https://openalex.org/C203005215 |
| concepts[7].level | 2 |
| concepts[7].score | 0.6067715287208557 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q79798 |
| concepts[7].display_name | Machine translation |
| concepts[8].id | https://openalex.org/C153924320 |
| concepts[8].level | 2 |
| concepts[8].score | 0.601473331451416 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q56475 |
| concepts[8].display_name | Hausa |
| concepts[9].id | https://openalex.org/C2777578687 |
| concepts[9].level | 2 |
| concepts[9].score | 0.5589425563812256 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q13218 |
| concepts[9].display_name | Xhosa |
| concepts[10].id | https://openalex.org/C192209626 |
| concepts[10].level | 2 |
| concepts[10].score | 0.5471979975700378 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q190909 |
| concepts[10].display_name | Focus (optics) |
| concepts[11].id | https://openalex.org/C157659113 |
| concepts[11].level | 2 |
| concepts[11].score | 0.47261834144592285 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q533822 |
| concepts[11].display_name | WordNet |
| concepts[12].id | https://openalex.org/C2776502983 |
| concepts[12].level | 2 |
| concepts[12].score | 0.44606438279151917 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q690182 |
| concepts[12].display_name | Contrast (vision) |
| concepts[13].id | https://openalex.org/C206345919 |
| concepts[13].level | 2 |
| concepts[13].score | 0.4386013150215149 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q20380951 |
| concepts[13].display_name | Resource (disambiguation) |
| concepts[14].id | https://openalex.org/C41895202 |
| concepts[14].level | 1 |
| concepts[14].score | 0.34677523374557495 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[14].display_name | Linguistics |
| concepts[15].id | https://openalex.org/C31258907 |
| concepts[15].level | 1 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q1301371 |
| concepts[15].display_name | Computer network |
| concepts[16].id | https://openalex.org/C162324750 |
| concepts[16].level | 0 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q8134 |
| concepts[16].display_name | Economics |
| concepts[17].id | https://openalex.org/C187736073 |
| concepts[17].level | 1 |
| concepts[17].score | 0.0 |
| concepts[17].wikidata | https://www.wikidata.org/wiki/Q2920921 |
| concepts[17].display_name | Management |
| concepts[18].id | https://openalex.org/C121332964 |
| concepts[18].level | 0 |
| concepts[18].score | 0.0 |
| concepts[18].wikidata | https://www.wikidata.org/wiki/Q413 |
| concepts[18].display_name | Physics |
| concepts[19].id | https://openalex.org/C120665830 |
| concepts[19].level | 1 |
| concepts[19].score | 0.0 |
| concepts[19].wikidata | https://www.wikidata.org/wiki/Q14620 |
| concepts[19].display_name | Optics |
| concepts[20].id | https://openalex.org/C138885662 |
| concepts[20].level | 0 |
| concepts[20].score | 0.0 |
| concepts[20].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[20].display_name | Philosophy |
| keywords[0].id | https://openalex.org/keywords/bengali |
| keywords[0].score | 0.8861085772514343 |
| keywords[0].display_name | Bengali |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.7849254608154297 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/zulu |
| keywords[2].score | 0.771683931350708 |
| keywords[2].display_name | Zulu |
| keywords[3].id | https://openalex.org/keywords/task |
| keywords[3].score | 0.6971642971038818 |
| keywords[3].display_name | Task (project management) |
| keywords[4].id | https://openalex.org/keywords/natural-language-processing |
| keywords[4].score | 0.6872802376747131 |
| keywords[4].display_name | Natural language processing |
| keywords[5].id | https://openalex.org/keywords/hindi |
| keywords[5].score | 0.6831809282302856 |
| keywords[5].display_name | Hindi |
| keywords[6].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[6].score | 0.6371247172355652 |
| keywords[6].display_name | Artificial intelligence |
| keywords[7].id | https://openalex.org/keywords/machine-translation |
| keywords[7].score | 0.6067715287208557 |
| keywords[7].display_name | Machine translation |
| keywords[8].id | https://openalex.org/keywords/hausa |
| keywords[8].score | 0.601473331451416 |
| keywords[8].display_name | Hausa |
| keywords[9].id | https://openalex.org/keywords/xhosa |
| keywords[9].score | 0.5589425563812256 |
| keywords[9].display_name | Xhosa |
| keywords[10].id | https://openalex.org/keywords/focus |
| keywords[10].score | 0.5471979975700378 |
| keywords[10].display_name | Focus (optics) |
| keywords[11].id | https://openalex.org/keywords/wordnet |
| keywords[11].score | 0.47261834144592285 |
| keywords[11].display_name | WordNet |
| keywords[12].id | https://openalex.org/keywords/contrast |
| keywords[12].score | 0.44606438279151917 |
| keywords[12].display_name | Contrast (vision) |
| keywords[13].id | https://openalex.org/keywords/resource |
| keywords[13].score | 0.4386013150215149 |
| keywords[13].display_name | Resource (disambiguation) |
| keywords[14].id | https://openalex.org/keywords/linguistics |
| keywords[14].score | 0.34677523374557495 |
| keywords[14].display_name | Linguistics |
| language | |
| locations[0].id | pmh:oai:arXiv.org:2111.13284 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2111.13284 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2111.13284 |
| indexed_in | arxiv |
| authorships[0].author.id | https://openalex.org/A5007758583 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Amr Hendy |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Hendy, Amr |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5064043933 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | Esraa A. Gad |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Gad, Esraa A. |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101190269 |
| authorships[2].author.orcid | |
| authorships[2].author.display_name | Mohamed Abdelghaffar |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Abdelghaffar, Mohamed |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5020195877 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Jailan S. ElMosalami |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | ElMosalami, Jailan S. |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5021938376 |
| authorships[4].author.orcid | https://orcid.org/0000-0002-4445-9767 |
| authorships[4].author.display_name | Mohamed Afify |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Afify, Mohamed |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5048873489 |
| authorships[5].author.orcid | https://orcid.org/0000-0003-3561-3248 |
| authorships[5].author.display_name | Ahmed Y. Tawfik |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Tawfik, Ahmed Y. |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5030937723 |
| authorships[6].author.orcid | |
| authorships[6].author.display_name | Hany Hassan Awadalla |
| authorships[6].author_position | last |
| authorships[6].raw_author_name | Awadalla, Hany Hassan |
| authorships[6].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2111.13284 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2022-07-25T00:00:00 |
| display_name | Ensembling of Distilled Models from Multi-task Teachers for Constrained\n Resource Language Pairs |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T03:46:38.306776 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9990000128746033 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W4246639531, https://openalex.org/W312867150, https://openalex.org/W2588238902, https://openalex.org/W4238714294, https://openalex.org/W2252383073, https://openalex.org/W2047081739, https://openalex.org/W375194962, https://openalex.org/W2121631514, https://openalex.org/W2143928726, https://openalex.org/W397957968 |
| cited_by_count | 0 |
| locations_count | 1 |
| best_oa_location.id | pmh:oai:arXiv.org:2111.13284 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2111.13284 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2111.13284 |
| primary_location.id | pmh:oai:arXiv.org:2111.13284 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2111.13284 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2111.13284 |
| publication_date | 2021-11-25 |
| publication_year | 2021 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 52, 71 |
| abstract_inverted_index.In | 60 |
| abstract_inverted_index.To | 38 |
| abstract_inverted_index.We | 14 |
| abstract_inverted_index.in | 92 |
| abstract_inverted_index.of | 9, 42 |
| abstract_inverted_index.on | 16 |
| abstract_inverted_index.to | 5, 24, 29, 35, 97, 108, 118 |
| abstract_inverted_index.we | 47, 62 |
| abstract_inverted_index.25% | 102 |
| abstract_inverted_index.70% | 89 |
| abstract_inverted_index.and | 25, 30, 33, 77, 98, 109, 112, 114 |
| abstract_inverted_index.for | 95, 105 |
| abstract_inverted_index.low | 20, 44 |
| abstract_inverted_index.our | 3 |
| abstract_inverted_index.the | 6, 17, 40, 64, 81 |
| abstract_inverted_index.two | 82 |
| abstract_inverted_index.BLEU | 93 |
| abstract_inverted_index.This | 0 |
| abstract_inverted_index.Zulu | 116 |
| abstract_inverted_index.back | 67, 75 |
| abstract_inverted_index.both | 56, 106 |
| abstract_inverted_index.data | 46, 65 |
| abstract_inverted_index.from | 26, 31, 99, 110, 115 |
| abstract_inverted_index.gain | 91 |
| abstract_inverted_index.then | 79 |
| abstract_inverted_index.WMT21 | 10 |
| abstract_inverted_index.Xhosa | 34 |
| abstract_inverted_index.Zulu. | 37 |
| abstract_inverted_index.data. | 59 |
| abstract_inverted_index.focus | 15 |
| abstract_inverted_index.model | 50, 73 |
| abstract_inverted_index.paper | 1 |
| abstract_inverted_index.point | 94 |
| abstract_inverted_index.task. | 13 |
| abstract_inverted_index.three | 18 |
| abstract_inverted_index.track | 8 |
| abstract_inverted_index.train | 70 |
| abstract_inverted_index.using | 51, 66, 84 |
| abstract_inverted_index.Hausa, | 32, 100 |
| abstract_inverted_index.Hindi, | 27, 111 |
| abstract_inverted_index.around | 88 |
| abstract_inverted_index.models | 83 |
| abstract_inverted_index.Bengali | 23, 107 |
| abstract_inverted_index.English | 28, 96 |
| abstract_inverted_index.We\nsee | 87 |
| abstract_inverted_index.augment | 63 |
| abstract_inverted_index.combine | 80 |
| abstract_inverted_index.We\nalso | 69 |
| abstract_inverted_index.compared | 117 |
| abstract_inverted_index.mapping. | 86 |
| abstract_inverted_index.overcome | 39 |
| abstract_inverted_index.parallel | 45, 57 |
| abstract_inverted_index.relative | 90, 103 |
| abstract_inverted_index.resource | 21 |
| abstract_inverted_index.train\na | 48 |
| abstract_inverted_index.Xhosa\nto | 113 |
| abstract_inverted_index.addition, | 61 |
| abstract_inverted_index.and\nfrom | 36 |
| abstract_inverted_index.bilingual | 72, 119 |
| abstract_inverted_index.describes | 2 |
| abstract_inverted_index.employing | 55 |
| abstract_inverted_index.multitask | 53 |
| abstract_inverted_index.objective | 54 |
| abstract_inverted_index.limitation | 41 |
| abstract_inverted_index.relatively | 19, 43 |
| abstract_inverted_index.submission | 4 |
| abstract_inverted_index.and\naround | 101 |
| abstract_inverted_index.constrained | 7 |
| abstract_inverted_index.translation | 12, 76 |
| abstract_inverted_index.baselines.\n | 120 |
| abstract_inverted_index.improvements | 104 |
| abstract_inverted_index.multilingual | 49 |
| abstract_inverted_index.shared\nnews | 11 |
| abstract_inverted_index.translation. | 68 |
| abstract_inverted_index.incorporating | 74 |
| abstract_inverted_index.language\npairs | 22 |
| abstract_inverted_index.and\nmonolingual | 58 |
| abstract_inverted_index.sequence-to-sequence | 85 |
| abstract_inverted_index.knowledge\ndistillation | 78 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 7 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.8500000238418579 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile.value | 0.23427501 |
| citation_normalized_percentile.is_in_top_1_percent | False |
| citation_normalized_percentile.is_in_top_10_percent | False |