Matt Post
YOU?
Author Swipe
View article: CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation
CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation Open
Models for streaming speech translation (ST) can achieve high accuracy and low latency if they're developed with vast amounts of paired audio in the source language and written text in the target language. Yet, these text labels for the ta…
View article: PyMarian: Fast Neural Machine Translation and Evaluation in Python
PyMarian: Fast Neural Machine Translation and Evaluation in Python Open
The deep learning language of choice these days is Python; measured by factors such as available libraries and technical support, it is hard to beat. At the same time, software written in lower-level programming languages like C++ retain a…
View article: Recovering document annotations for sentence-level bitext
Recovering document annotations for sentence-level bitext Open
Data availability limits the scope of any given task. In machine translation, historical models were incapable of handling longer contexts, so the lack of document-level datasets was less noticeable. Now, despite the emergence of long-sequ…
View article: Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies
Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies Open
Ten years ago a single metric, BLEU, governed progress in machine translation research. For better or worse, there is no such consensus today, and consequently it is difficult for researchers to develop and retain the kinds of heuristic in…
View article: Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context
Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context Open
Lexical ambiguity is a challenging and pervasive problem in machine translation (\mt). We introduce a simple and scalable approach to resolve translation ambiguity by incorporating a small amount of extra-sentential context in neural \mt. …
View article: Identifying Context-Dependent Translations for Evaluation Set Production
Identifying Context-Dependent Translations for Evaluation Set Production Open
A major impediment to the transition to context-aware machine translation is the absence of good evaluation metrics and test sets. Sentences that require context to be translated correctly are rare in test sets, reducing the utility of sta…
View article: SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window
SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window Open
Reference-based metrics that operate at the sentence-level typically outperform quality estimation metrics, which have access only to the source and system output. This is unsurprising, since references resolve ambiguities that may be pres…
View article: SOTASTREAM: A Streaming Approach to Machine Translation Training
SOTASTREAM: A Streaming Approach to Machine Translation Training Open
Many machine translation toolkits make use of a data preparation step wherein raw data is transformed into a tensor format that can be used directly by the trainer. This preparation step is increasingly at odds with modern research and dev…
View article: Do GPTs Produce Less Literal Translations?
Do GPTs Produce Less Literal Translations? Open
Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose language models capable of addressing many natural language generation or understanding tasks. On the task of Machine Translation (MT), multiple works have investig…
View article: Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer
Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer Open
We introduce and demonstrate how to effectively train multilingual machine translation models with pixel representations. We experiment with two different data settings with a variety of language and script coverage, demonstrating improved…
View article: Escaping the sentence-level paradigm in machine translation
Escaping the sentence-level paradigm in machine translation Open
It is well-known that document context is vital for resolving a range of translation ambiguities, and in fact the document setting is the most natural setting for nearly all translation. It is therefore unfortunate that machine translation…
View article: Two Decades of the ACL Anthology: Development, Impact, and Open Challenges
Two Decades of the ACL Anthology: Development, Impact, and Open Challenges Open
The ACL Anthology is a prime resource for research papers within computational linguistics and natural language processing, while continuing to be an open-source and community-driven project. Since Gildea et al. (2018) reported on its stat…
View article: Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer
Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer Open
Pretrained multilingual translation models using either pixel or subword (bpe) representations trained on the many-to-one parallel TED-59 dataset, accompanying the EMNLP'23 paper "Multilingual Pixel Representations for Translation and Effe…
View article: Identifying Context-Dependent Translations for Evaluation Set Production
Identifying Context-Dependent Translations for Evaluation Set Production Open
A major impediment to the transition to contextual machine translation is the absence of good evaluation metrics and test sets. Sentences that require context to be translated correctly are rare in test sets, reducing the utility of standa…
View article: Evaluating Metrics for Document-context Evaluation in Machine Translation
Evaluating Metrics for Document-context Evaluation in Machine Translation Open
We describe our submission of a new metric, SLIDE (Raunak et al., 2023), to the WMT 2023 metrics task. SLIDE is a reference-free quality-estimation metric that works by constructing a fixed sentence-length window over the documents in a te…
View article: Do GPTs Produce Less Literal Translations?
Do GPTs Produce Less Literal Translations? Open
Large Language Models (LLMs) such as GPT-3 have emerged as general-purpose language models capable of addressing many natural language generation or understanding tasks. On the task of Machine Translation (MT), multiple works have investig…
View article: SOTASTREAM: A Streaming Approach to Machine Translation Training
SOTASTREAM: A Streaming Approach to Machine Translation Training Open
Matt Post, Thamme Gowda, Roman Grundkiewicz, Huda Khayrallah, Rohit Jain, Marcin Junczys-Dowmunt. Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023). 2023.
View article: Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models
Operationalizing Specifications, In Addition to Test Sets for Evaluating Constrained Generative Models Open
In this work, we present some recommendations on the evaluation of state-of-the-art generative models for constrained generation tasks. The progress on generative models has been rapid in recent years. These large-scale models have had thr…
View article: Additive Interventions Yield Robust Multi-Domain Machine Translation Models
Additive Interventions Yield Robust Multi-Domain Machine Translation Models Open
Additive interventions are a recently-proposed mechanism for controlling target-side attributes in neural machine translation. In contrast to tag-based approaches which manipulate the raw source sequence, interventions work by directly mod…
View article: SALTED: A Framework for SAlient Long-Tail Translation Error Detection
SALTED: A Framework for SAlient Long-Tail Translation Error Detection Open
Traditional machine translation (MT) metrics provide an average measure of translation quality that is insensitive to the long tail of behavioral problems in MT. Examples include translation of numbers, physical units, dropped content and …
View article: Large-Scale Streaming End-to-End Speech Translation with Neural Transducers
Large-Scale Streaming End-to-End Speech Translation with Neural Transducers Open
Neural transducers have been widely used in automatic speech recognition (ASR). In this paper, we introduce it to streaming end-to-end speech translation (ST), which aims to convert audio signals to texts in other languages directly. Compa…
View article: SALTED: A Framework for SAlient Long-tail Translation Error Detection
SALTED: A Framework for SAlient Long-tail Translation Error Detection Open
Traditional machine translation (MT) metrics provide an average measure of translation quality that is insensitive to the long tail of behavioral problems. Examples include translation of numbers, physical units, dropped content and halluc…
View article: The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task
The JHU-Microsoft Submission for WMT21 Quality Estimation Shared Task Open
This paper presents the JHU-Microsoft joint submission for WMT 2021 quality estimation shared task. We only participate in Task 2 (post-editing effort estimation) of the shared task, focusing on the target-side word-level quality estimatio…