Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book? Article Swipe
Seth Aycock
,
David Stap
,
Di Wu
,
Christof Monz
,
Khalil Sima’an
·
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2409.19151
YOU?
·
· 2024
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2409.19151
Extremely low-resource (XLR) languages lack substantial corpora for training NLP models, motivating the use of all available resources such as dictionaries and grammar books. Machine Translation from One Book (Tanzer et al., 2024) suggests that prompting long-context LLMs with one grammar book enables English-Kalamang translation, an XLR language unseen by LLMs - a noteworthy case of linguistics helping an NLP task. We investigate the source of this translation ability, finding almost all improvements stem from the book's parallel examples rather than its grammatical explanations. We find similar results for Nepali and Guarani, seen low-resource languages, and we achieve performance comparable to an LLM with a grammar book by simply fine-tuning an encoder-decoder translation model. We then investigate where grammar books help by testing two linguistic tasks, grammaticality judgment and gloss prediction, and we explore what kind of grammatical knowledge helps by introducing a typological feature prompt that achieves leading results on these more relevant tasks. We thus emphasise the importance of task-appropriate data for XLR languages: parallel examples for translation, and grammatical data for linguistic tasks. As we find no evidence that long-context LLMs can make effective use of grammatical explanations for XLR translation, we conclude data collection for multilingual XLR tasks such as translation is best focused on parallel data over linguistic description.
Related Topics
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2409.19151
- https://arxiv.org/pdf/2409.19151
- OA Status
- green
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4403810484
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4403810484Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2409.19151Digital Object Identifier
- Title
-
Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?Work title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2024Year of publication
- Publication date
-
2024-09-27Full publication date if available
- Authors
-
Seth Aycock, David Stap, Di Wu, Christof Monz, Khalil Sima’anList of authors in order
- Landing page
-
https://arxiv.org/abs/2409.19151Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2409.19151Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2409.19151Direct OA link when available
- Concepts
-
Grammar, Linguistics, Resource (disambiguation), Computer science, Philosophy, Computer networkTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
0Total citation count in OpenAlex
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4403810484 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2409.19151 |
| ids.doi | https://doi.org/10.48550/arxiv.2409.19151 |
| ids.openalex | https://openalex.org/W4403810484 |
| fwci | |
| type | preprint |
| title | Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book? |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9986000061035156 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T13523 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9175999760627747 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1703 |
| topics[1].subfield.display_name | Computational Theory and Mathematics |
| topics[1].display_name | Mathematics, Computing, and Information Processing |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C26022165 |
| concepts[0].level | 2 |
| concepts[0].score | 0.6583162546157837 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q8091 |
| concepts[0].display_name | Grammar |
| concepts[1].id | https://openalex.org/C41895202 |
| concepts[1].level | 1 |
| concepts[1].score | 0.5520270466804504 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[1].display_name | Linguistics |
| concepts[2].id | https://openalex.org/C206345919 |
| concepts[2].level | 2 |
| concepts[2].score | 0.4680231511592865 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q20380951 |
| concepts[2].display_name | Resource (disambiguation) |
| concepts[3].id | https://openalex.org/C41008148 |
| concepts[3].level | 0 |
| concepts[3].score | 0.4498436450958252 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[3].display_name | Computer science |
| concepts[4].id | https://openalex.org/C138885662 |
| concepts[4].level | 0 |
| concepts[4].score | 0.14998304843902588 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[4].display_name | Philosophy |
| concepts[5].id | https://openalex.org/C31258907 |
| concepts[5].level | 1 |
| concepts[5].score | 0.0 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q1301371 |
| concepts[5].display_name | Computer network |
| keywords[0].id | https://openalex.org/keywords/grammar |
| keywords[0].score | 0.6583162546157837 |
| keywords[0].display_name | Grammar |
| keywords[1].id | https://openalex.org/keywords/linguistics |
| keywords[1].score | 0.5520270466804504 |
| keywords[1].display_name | Linguistics |
| keywords[2].id | https://openalex.org/keywords/resource |
| keywords[2].score | 0.4680231511592865 |
| keywords[2].display_name | Resource (disambiguation) |
| keywords[3].id | https://openalex.org/keywords/computer-science |
| keywords[3].score | 0.4498436450958252 |
| keywords[3].display_name | Computer science |
| keywords[4].id | https://openalex.org/keywords/philosophy |
| keywords[4].score | 0.14998304843902588 |
| keywords[4].display_name | Philosophy |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2409.19151 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2409.19151 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2409.19151 |
| locations[1].id | doi:10.48550/arxiv.2409.19151 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2409.19151 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5114424419 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Seth Aycock |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Aycock, Seth |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5015155057 |
| authorships[1].author.orcid | |
| authorships[1].author.display_name | David Stap |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Stap, David |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5101577317 |
| authorships[2].author.orcid | https://orcid.org/0000-0003-1664-5893 |
| authorships[2].author.display_name | Di Wu |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Wu, Di |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5109059955 |
| authorships[3].author.orcid | |
| authorships[3].author.display_name | Christof Monz |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Monz, Christof |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5034829413 |
| authorships[4].author.orcid | |
| authorships[4].author.display_name | Khalil Sima’an |
| authorships[4].author_position | last |
| authorships[4].raw_author_name | Sima'an, Khalil |
| authorships[4].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2409.19151 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book? |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9986000061035156 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W4391375266, https://openalex.org/W2899084033, https://openalex.org/W2748952813, https://openalex.org/W2390279801, https://openalex.org/W4391913857, https://openalex.org/W2358668433, https://openalex.org/W4396701345, https://openalex.org/W2376932109, https://openalex.org/W2001405890, https://openalex.org/W4396696052 |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2409.19151 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2409.19151 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2409.19151 |
| primary_location.id | pmh:oai:arXiv.org:2409.19151 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2409.19151 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2409.19151 |
| publication_date | 2024-09-27 |
| publication_year | 2024 |
| referenced_works_count | 0 |
| abstract_inverted_index.- | 51 |
| abstract_inverted_index.a | 52, 104, 142 |
| abstract_inverted_index.As | 176 |
| abstract_inverted_index.We | 61, 84, 114, 155 |
| abstract_inverted_index.an | 45, 58, 101, 110 |
| abstract_inverted_index.as | 19, 203 |
| abstract_inverted_index.by | 49, 107, 121, 140 |
| abstract_inverted_index.et | 30 |
| abstract_inverted_index.is | 205 |
| abstract_inverted_index.no | 179 |
| abstract_inverted_index.of | 14, 55, 65, 136, 160, 188 |
| abstract_inverted_index.on | 150, 208 |
| abstract_inverted_index.to | 100 |
| abstract_inverted_index.we | 96, 132, 177, 194 |
| abstract_inverted_index.LLM | 102 |
| abstract_inverted_index.NLP | 9, 59 |
| abstract_inverted_index.One | 27 |
| abstract_inverted_index.XLR | 46, 164, 192, 200 |
| abstract_inverted_index.all | 15, 71 |
| abstract_inverted_index.and | 21, 90, 95, 128, 131, 170 |
| abstract_inverted_index.can | 184 |
| abstract_inverted_index.for | 7, 88, 163, 168, 173, 191, 198 |
| abstract_inverted_index.its | 81 |
| abstract_inverted_index.one | 39 |
| abstract_inverted_index.the | 12, 63, 75, 158 |
| abstract_inverted_index.two | 123 |
| abstract_inverted_index.use | 13, 187 |
| abstract_inverted_index.Book | 28 |
| abstract_inverted_index.LLMs | 37, 50, 183 |
| abstract_inverted_index.al., | 31 |
| abstract_inverted_index.best | 206 |
| abstract_inverted_index.book | 41, 106 |
| abstract_inverted_index.case | 54 |
| abstract_inverted_index.data | 162, 172, 196, 210 |
| abstract_inverted_index.find | 85, 178 |
| abstract_inverted_index.from | 26, 74 |
| abstract_inverted_index.help | 120 |
| abstract_inverted_index.kind | 135 |
| abstract_inverted_index.lack | 4 |
| abstract_inverted_index.make | 185 |
| abstract_inverted_index.more | 152 |
| abstract_inverted_index.over | 211 |
| abstract_inverted_index.seen | 92 |
| abstract_inverted_index.stem | 73 |
| abstract_inverted_index.such | 18, 202 |
| abstract_inverted_index.than | 80 |
| abstract_inverted_index.that | 34, 146, 181 |
| abstract_inverted_index.then | 115 |
| abstract_inverted_index.this | 66 |
| abstract_inverted_index.thus | 156 |
| abstract_inverted_index.what | 134 |
| abstract_inverted_index.with | 38, 103 |
| abstract_inverted_index.(XLR) | 2 |
| abstract_inverted_index.2024) | 32 |
| abstract_inverted_index.books | 119 |
| abstract_inverted_index.gloss | 129 |
| abstract_inverted_index.helps | 139 |
| abstract_inverted_index.task. | 60 |
| abstract_inverted_index.tasks | 201 |
| abstract_inverted_index.these | 151 |
| abstract_inverted_index.where | 117 |
| abstract_inverted_index.Nepali | 89 |
| abstract_inverted_index.almost | 70 |
| abstract_inverted_index.book's | 76 |
| abstract_inverted_index.books. | 23 |
| abstract_inverted_index.model. | 113 |
| abstract_inverted_index.prompt | 145 |
| abstract_inverted_index.rather | 79 |
| abstract_inverted_index.simply | 108 |
| abstract_inverted_index.source | 64 |
| abstract_inverted_index.tasks, | 125 |
| abstract_inverted_index.tasks. | 154, 175 |
| abstract_inverted_index.unseen | 48 |
| abstract_inverted_index.(Tanzer | 29 |
| abstract_inverted_index.Machine | 24 |
| abstract_inverted_index.achieve | 97 |
| abstract_inverted_index.corpora | 6 |
| abstract_inverted_index.enables | 42 |
| abstract_inverted_index.explore | 133 |
| abstract_inverted_index.feature | 144 |
| abstract_inverted_index.finding | 69 |
| abstract_inverted_index.focused | 207 |
| abstract_inverted_index.grammar | 22, 40, 105, 118 |
| abstract_inverted_index.helping | 57 |
| abstract_inverted_index.leading | 148 |
| abstract_inverted_index.models, | 10 |
| abstract_inverted_index.results | 87, 149 |
| abstract_inverted_index.similar | 86 |
| abstract_inverted_index.testing | 122 |
| abstract_inverted_index.Guarani, | 91 |
| abstract_inverted_index.ability, | 68 |
| abstract_inverted_index.achieves | 147 |
| abstract_inverted_index.conclude | 195 |
| abstract_inverted_index.evidence | 180 |
| abstract_inverted_index.examples | 78, 167 |
| abstract_inverted_index.judgment | 127 |
| abstract_inverted_index.language | 47 |
| abstract_inverted_index.parallel | 77, 166, 209 |
| abstract_inverted_index.relevant | 153 |
| abstract_inverted_index.suggests | 33 |
| abstract_inverted_index.training | 8 |
| abstract_inverted_index.Extremely | 0 |
| abstract_inverted_index.available | 16 |
| abstract_inverted_index.effective | 186 |
| abstract_inverted_index.emphasise | 157 |
| abstract_inverted_index.knowledge | 138 |
| abstract_inverted_index.languages | 3 |
| abstract_inverted_index.prompting | 35 |
| abstract_inverted_index.resources | 17 |
| abstract_inverted_index.collection | 197 |
| abstract_inverted_index.comparable | 99 |
| abstract_inverted_index.importance | 159 |
| abstract_inverted_index.languages, | 94 |
| abstract_inverted_index.languages: | 165 |
| abstract_inverted_index.linguistic | 124, 174, 212 |
| abstract_inverted_index.motivating | 11 |
| abstract_inverted_index.noteworthy | 53 |
| abstract_inverted_index.Translation | 25 |
| abstract_inverted_index.fine-tuning | 109 |
| abstract_inverted_index.grammatical | 82, 137, 171, 189 |
| abstract_inverted_index.introducing | 141 |
| abstract_inverted_index.investigate | 62, 116 |
| abstract_inverted_index.linguistics | 56 |
| abstract_inverted_index.performance | 98 |
| abstract_inverted_index.prediction, | 130 |
| abstract_inverted_index.substantial | 5 |
| abstract_inverted_index.translation | 67, 112, 204 |
| abstract_inverted_index.typological | 143 |
| abstract_inverted_index.description. | 213 |
| abstract_inverted_index.dictionaries | 20 |
| abstract_inverted_index.explanations | 190 |
| abstract_inverted_index.improvements | 72 |
| abstract_inverted_index.long-context | 36, 182 |
| abstract_inverted_index.low-resource | 1, 93 |
| abstract_inverted_index.multilingual | 199 |
| abstract_inverted_index.translation, | 44, 169, 193 |
| abstract_inverted_index.explanations. | 83 |
| abstract_inverted_index.grammaticality | 126 |
| abstract_inverted_index.encoder-decoder | 111 |
| abstract_inverted_index.English-Kalamang | 43 |
| abstract_inverted_index.task-appropriate | 161 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 5 |
| citation_normalized_percentile |