Word Alignment in the Era of Deep Learning: A Tutorial Article Swipe
The word alignment task, despite its prominence in the era of statistical machine translation (SMT), is niche and under-explored today. In this two-part tutorial, we argue for the continued relevance for word alignment. The first part provides a historical background to word alignment as a core component of the traditional SMT pipeline. We zero-in on GIZA++, an unsupervised, statistical word aligner with surprising longevity. Jumping forward to the era of neural machine translation (NMT), we show how insights from word alignment inspired the attention mechanism fundamental to present-day NMT. The second part shifts to a survey approach. We cover neural word aligners, showing the slow but steady progress towards surpassing GIZA++ performance. Finally, we cover the present-day applications of word alignment, from cross-lingual annotation projection, to improving translation.
Related Topics
Concepts
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2212.00138
- https://arxiv.org/pdf/2212.00138
- OA Status
- green
- Cited By
- 2
- Related Works
- 10
- OpenAlex ID
- https://openalex.org/W4310628596
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4310628596Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2212.00138Digital Object Identifier
- Title
-
Word Alignment in the Era of Deep Learning: A TutorialWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2022Year of publication
- Publication date
-
2022-11-30Full publication date if available
- Authors
-
Bryan LiList of authors in order
- Landing page
-
https://arxiv.org/abs/2212.00138Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2212.00138Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2212.00138Direct OA link when available
- Concepts
-
Word (group theory), Computer science, Machine translation, Artificial intelligence, Natural language processing, Pipeline (software), Translation (biology), Relevance (law), Linguistics, Chemistry, Law, Messenger RNA, Biochemistry, Gene, Programming language, Political science, PhilosophyTop concepts (fields/topics) attached by OpenAlex
- Cited by
-
2Total citation count in OpenAlex
- Citations by year (recent)
-
2024: 1, 2023: 1Per-year citation counts (last 5 years)
- Related works (count)
-
10Other works algorithmically related by OpenAlex
Full payload
| id | https://openalex.org/W4310628596 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2212.00138 |
| ids.doi | https://doi.org/10.48550/arxiv.2212.00138 |
| ids.openalex | https://openalex.org/W4310628596 |
| fwci | |
| type | preprint |
| title | Word Alignment in the Era of Deep Learning: A Tutorial |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T10181 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.9998000264167786 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Natural Language Processing Techniques |
| topics[1].id | https://openalex.org/T10028 |
| topics[1].field.id | https://openalex.org/fields/17 |
| topics[1].field.display_name | Computer Science |
| topics[1].score | 0.9959999918937683 |
| topics[1].domain.id | https://openalex.org/domains/3 |
| topics[1].domain.display_name | Physical Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/1702 |
| topics[1].subfield.display_name | Artificial Intelligence |
| topics[1].display_name | Topic Modeling |
| topics[2].id | https://openalex.org/T10201 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.9175999760627747 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Speech Recognition and Synthesis |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| concepts[0].id | https://openalex.org/C90805587 |
| concepts[0].level | 2 |
| concepts[0].score | 0.7841670513153076 |
| concepts[0].wikidata | https://www.wikidata.org/wiki/Q10944557 |
| concepts[0].display_name | Word (group theory) |
| concepts[1].id | https://openalex.org/C41008148 |
| concepts[1].level | 0 |
| concepts[1].score | 0.7713254690170288 |
| concepts[1].wikidata | https://www.wikidata.org/wiki/Q21198 |
| concepts[1].display_name | Computer science |
| concepts[2].id | https://openalex.org/C203005215 |
| concepts[2].level | 2 |
| concepts[2].score | 0.6877633333206177 |
| concepts[2].wikidata | https://www.wikidata.org/wiki/Q79798 |
| concepts[2].display_name | Machine translation |
| concepts[3].id | https://openalex.org/C154945302 |
| concepts[3].level | 1 |
| concepts[3].score | 0.6550266146659851 |
| concepts[3].wikidata | https://www.wikidata.org/wiki/Q11660 |
| concepts[3].display_name | Artificial intelligence |
| concepts[4].id | https://openalex.org/C204321447 |
| concepts[4].level | 1 |
| concepts[4].score | 0.640919029712677 |
| concepts[4].wikidata | https://www.wikidata.org/wiki/Q30642 |
| concepts[4].display_name | Natural language processing |
| concepts[5].id | https://openalex.org/C43521106 |
| concepts[5].level | 2 |
| concepts[5].score | 0.6196942329406738 |
| concepts[5].wikidata | https://www.wikidata.org/wiki/Q2165493 |
| concepts[5].display_name | Pipeline (software) |
| concepts[6].id | https://openalex.org/C149364088 |
| concepts[6].level | 4 |
| concepts[6].score | 0.5564197897911072 |
| concepts[6].wikidata | https://www.wikidata.org/wiki/Q185917 |
| concepts[6].display_name | Translation (biology) |
| concepts[7].id | https://openalex.org/C158154518 |
| concepts[7].level | 2 |
| concepts[7].score | 0.4242505431175232 |
| concepts[7].wikidata | https://www.wikidata.org/wiki/Q7310970 |
| concepts[7].display_name | Relevance (law) |
| concepts[8].id | https://openalex.org/C41895202 |
| concepts[8].level | 1 |
| concepts[8].score | 0.212671160697937 |
| concepts[8].wikidata | https://www.wikidata.org/wiki/Q8162 |
| concepts[8].display_name | Linguistics |
| concepts[9].id | https://openalex.org/C185592680 |
| concepts[9].level | 0 |
| concepts[9].score | 0.0 |
| concepts[9].wikidata | https://www.wikidata.org/wiki/Q2329 |
| concepts[9].display_name | Chemistry |
| concepts[10].id | https://openalex.org/C199539241 |
| concepts[10].level | 1 |
| concepts[10].score | 0.0 |
| concepts[10].wikidata | https://www.wikidata.org/wiki/Q7748 |
| concepts[10].display_name | Law |
| concepts[11].id | https://openalex.org/C105580179 |
| concepts[11].level | 3 |
| concepts[11].score | 0.0 |
| concepts[11].wikidata | https://www.wikidata.org/wiki/Q188928 |
| concepts[11].display_name | Messenger RNA |
| concepts[12].id | https://openalex.org/C55493867 |
| concepts[12].level | 1 |
| concepts[12].score | 0.0 |
| concepts[12].wikidata | https://www.wikidata.org/wiki/Q7094 |
| concepts[12].display_name | Biochemistry |
| concepts[13].id | https://openalex.org/C104317684 |
| concepts[13].level | 2 |
| concepts[13].score | 0.0 |
| concepts[13].wikidata | https://www.wikidata.org/wiki/Q7187 |
| concepts[13].display_name | Gene |
| concepts[14].id | https://openalex.org/C199360897 |
| concepts[14].level | 1 |
| concepts[14].score | 0.0 |
| concepts[14].wikidata | https://www.wikidata.org/wiki/Q9143 |
| concepts[14].display_name | Programming language |
| concepts[15].id | https://openalex.org/C17744445 |
| concepts[15].level | 0 |
| concepts[15].score | 0.0 |
| concepts[15].wikidata | https://www.wikidata.org/wiki/Q36442 |
| concepts[15].display_name | Political science |
| concepts[16].id | https://openalex.org/C138885662 |
| concepts[16].level | 0 |
| concepts[16].score | 0.0 |
| concepts[16].wikidata | https://www.wikidata.org/wiki/Q5891 |
| concepts[16].display_name | Philosophy |
| keywords[0].id | https://openalex.org/keywords/word |
| keywords[0].score | 0.7841670513153076 |
| keywords[0].display_name | Word (group theory) |
| keywords[1].id | https://openalex.org/keywords/computer-science |
| keywords[1].score | 0.7713254690170288 |
| keywords[1].display_name | Computer science |
| keywords[2].id | https://openalex.org/keywords/machine-translation |
| keywords[2].score | 0.6877633333206177 |
| keywords[2].display_name | Machine translation |
| keywords[3].id | https://openalex.org/keywords/artificial-intelligence |
| keywords[3].score | 0.6550266146659851 |
| keywords[3].display_name | Artificial intelligence |
| keywords[4].id | https://openalex.org/keywords/natural-language-processing |
| keywords[4].score | 0.640919029712677 |
| keywords[4].display_name | Natural language processing |
| keywords[5].id | https://openalex.org/keywords/pipeline |
| keywords[5].score | 0.6196942329406738 |
| keywords[5].display_name | Pipeline (software) |
| keywords[6].id | https://openalex.org/keywords/translation |
| keywords[6].score | 0.5564197897911072 |
| keywords[6].display_name | Translation (biology) |
| keywords[7].id | https://openalex.org/keywords/relevance |
| keywords[7].score | 0.4242505431175232 |
| keywords[7].display_name | Relevance (law) |
| keywords[8].id | https://openalex.org/keywords/linguistics |
| keywords[8].score | 0.212671160697937 |
| keywords[8].display_name | Linguistics |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2212.00138 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2212.00138 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2212.00138 |
| locations[1].id | doi:10.48550/arxiv.2212.00138 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2212.00138 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5085287152 |
| authorships[0].author.orcid | https://orcid.org/0000-0002-7972-9966 |
| authorships[0].author.display_name | Bryan Li |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Li, Bryan |
| authorships[0].is_corresponding | True |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2212.00138 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | Word Alignment in the Era of Deep Learning: A Tutorial |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T10181 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.9998000264167786 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Natural Language Processing Techniques |
| related_works | https://openalex.org/W3011059803, https://openalex.org/W2085384747, https://openalex.org/W2088166309, https://openalex.org/W1891216533, https://openalex.org/W2106071040, https://openalex.org/W4312133475, https://openalex.org/W4238976562, https://openalex.org/W3151736118, https://openalex.org/W2883671469, https://openalex.org/W2728761353 |
| cited_by_count | 2 |
| counts_by_year[0].year | 2024 |
| counts_by_year[0].cited_by_count | 1 |
| counts_by_year[1].year | 2023 |
| counts_by_year[1].cited_by_count | 1 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2212.00138 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2212.00138 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2212.00138 |
| primary_location.id | pmh:oai:arXiv.org:2212.00138 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2212.00138 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2212.00138 |
| publication_date | 2022-11-30 |
| publication_year | 2022 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 37, 44, 94 |
| abstract_inverted_index.In | 20 |
| abstract_inverted_index.We | 52, 97 |
| abstract_inverted_index.an | 56 |
| abstract_inverted_index.as | 43 |
| abstract_inverted_index.in | 7 |
| abstract_inverted_index.is | 15 |
| abstract_inverted_index.of | 10, 47, 69, 118 |
| abstract_inverted_index.on | 54 |
| abstract_inverted_index.to | 40, 66, 86, 93, 125 |
| abstract_inverted_index.we | 24, 74, 113 |
| abstract_inverted_index.SMT | 50 |
| abstract_inverted_index.The | 0, 33, 89 |
| abstract_inverted_index.and | 17 |
| abstract_inverted_index.but | 105 |
| abstract_inverted_index.era | 9, 68 |
| abstract_inverted_index.for | 26, 30 |
| abstract_inverted_index.how | 76 |
| abstract_inverted_index.its | 5 |
| abstract_inverted_index.the | 8, 27, 48, 67, 82, 103, 115 |
| abstract_inverted_index.NMT. | 88 |
| abstract_inverted_index.core | 45 |
| abstract_inverted_index.from | 78, 121 |
| abstract_inverted_index.part | 35, 91 |
| abstract_inverted_index.show | 75 |
| abstract_inverted_index.slow | 104 |
| abstract_inverted_index.this | 21 |
| abstract_inverted_index.with | 61 |
| abstract_inverted_index.word | 1, 31, 41, 59, 79, 100, 119 |
| abstract_inverted_index.argue | 25 |
| abstract_inverted_index.cover | 98, 114 |
| abstract_inverted_index.first | 34 |
| abstract_inverted_index.niche | 16 |
| abstract_inverted_index.task, | 3 |
| abstract_inverted_index.(NMT), | 73 |
| abstract_inverted_index.(SMT), | 14 |
| abstract_inverted_index.GIZA++ | 110 |
| abstract_inverted_index.neural | 70, 99 |
| abstract_inverted_index.second | 90 |
| abstract_inverted_index.shifts | 92 |
| abstract_inverted_index.steady | 106 |
| abstract_inverted_index.survey | 95 |
| abstract_inverted_index.today. | 19 |
| abstract_inverted_index.GIZA++, | 55 |
| abstract_inverted_index.Jumping | 64 |
| abstract_inverted_index.aligner | 60 |
| abstract_inverted_index.despite | 4 |
| abstract_inverted_index.forward | 65 |
| abstract_inverted_index.machine | 12, 71 |
| abstract_inverted_index.showing | 102 |
| abstract_inverted_index.towards | 108 |
| abstract_inverted_index.zero-in | 53 |
| abstract_inverted_index.Finally, | 112 |
| abstract_inverted_index.insights | 77 |
| abstract_inverted_index.inspired | 81 |
| abstract_inverted_index.progress | 107 |
| abstract_inverted_index.provides | 36 |
| abstract_inverted_index.two-part | 22 |
| abstract_inverted_index.aligners, | 101 |
| abstract_inverted_index.alignment | 2, 42, 80 |
| abstract_inverted_index.approach. | 96 |
| abstract_inverted_index.attention | 83 |
| abstract_inverted_index.component | 46 |
| abstract_inverted_index.continued | 28 |
| abstract_inverted_index.improving | 126 |
| abstract_inverted_index.mechanism | 84 |
| abstract_inverted_index.pipeline. | 51 |
| abstract_inverted_index.relevance | 29 |
| abstract_inverted_index.tutorial, | 23 |
| abstract_inverted_index.alignment, | 120 |
| abstract_inverted_index.alignment. | 32 |
| abstract_inverted_index.annotation | 123 |
| abstract_inverted_index.background | 39 |
| abstract_inverted_index.historical | 38 |
| abstract_inverted_index.longevity. | 63 |
| abstract_inverted_index.prominence | 6 |
| abstract_inverted_index.surpassing | 109 |
| abstract_inverted_index.surprising | 62 |
| abstract_inverted_index.fundamental | 85 |
| abstract_inverted_index.present-day | 87, 116 |
| abstract_inverted_index.projection, | 124 |
| abstract_inverted_index.statistical | 11, 58 |
| abstract_inverted_index.traditional | 49 |
| abstract_inverted_index.translation | 13, 72 |
| abstract_inverted_index.applications | 117 |
| abstract_inverted_index.performance. | 111 |
| abstract_inverted_index.translation. | 127 |
| abstract_inverted_index.cross-lingual | 122 |
| abstract_inverted_index.unsupervised, | 57 |
| abstract_inverted_index.under-explored | 18 |
| cited_by_percentile_year | |
| corresponding_author_ids | https://openalex.org/A5085287152 |
| countries_distinct_count | 0 |
| institutions_distinct_count | 1 |
| sustainable_development_goals[0].id | https://metadata.un.org/sdg/4 |
| sustainable_development_goals[0].score | 0.699999988079071 |
| sustainable_development_goals[0].display_name | Quality Education |
| citation_normalized_percentile |