David Stap
YOU?
Author Swipe
View article: The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation
The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation Open
Prior research diverges on language diversity in LLM fine-tuning: Some studies report benefits while others find no advantages. Through controlled fine-tuning experiments across 132 translation directions, we systematically resolve these d…
View article: MMTEB: Massive Multilingual Text Embedding Benchmark
MMTEB: Massive Multilingual Text Embedding Benchmark Open
Text embeddings are typically evaluated on a limited set of tasks, which are constrained by language, domain, and task diversity. To address these limitations and provide a more comprehensive evaluation, we introduce the Massive Multilingu…
View article: The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation
The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation Open
View article: Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book?
Can LLMs Really Learn to Translate a Low-Resource Language from One Grammar Book? Open
Extremely low-resource (XLR) languages lack substantial corpora for training NLP models, motivating the use of all available resources such as dictionaries and grammar books. Machine Translation from One Book (Tanzer et al., 2024) suggests…
View article: Data Contamination Report from the 2024 CONDA Shared Task
Data Contamination Report from the 2024 CONDA Shared Task Open
The 1st Workshop on Data Contamination (CONDA 2024) focuses on all relevant aspects of data contamination in natural language processing, where data contamination is understood as situations where evaluation data is included in pre-trainin…
View article: The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities
The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities Open
Fine-tuning large language models (LLMs) for machine translation has shown improvements in overall translation quality. However, it is unclear what is the impact of fine-tuning on desirable LLM behaviors that are not present in neural mach…
View article: How Far Can 100 Samples Go? Unlocking Overall Zero-Shot Multilingual Translation via Tiny Multi-Parallel Data
How Far Can 100 Samples Go? Unlocking Overall Zero-Shot Multilingual Translation via Tiny Multi-Parallel Data Open
Zero-shot translation aims to translate between language pairs not seen during training in Multilingual Machine Translation (MMT) and is largely considered an open problem. A common, albeit resource-consuming, solution is to add as many re…
View article: Multilingual k-Nearest-Neighbor Machine Translation
Multilingual k-Nearest-Neighbor Machine Translation Open
k-nearest-neighbor machine translation has demonstrated remarkable improvements in machine translation quality by creating a datastore of cached examples. However, these improvements have been limited to high-resource language pairs, with …
View article: UvA-MT's Participation in the WMT23 General Translation Shared Task
UvA-MT's Participation in the WMT23 General Translation Shared Task Open
This paper describes the UvA-MT's submission to the WMT 2023 shared task on general machine translation. We participate in the constrained track in two directions: English Hebrew. In this competition, we show that by using one model to ha…
View article: Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens
Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens Open
We argue that translation quality alone is not a sufficient metric for measuring knowledge transfer in multilingual neural machine translation. To support this claim, we introduce Representational Transfer Potential (RTP), which measures r…
View article: Multilingual k-Nearest-Neighbor Machine Translation-Nearest-Neighbor Machine Translation
Multilingual k-Nearest-Neighbor Machine Translation-Nearest-Neighbor Machine Translation Open
k-nearest-neighbor machine translation has demonstrated remarkable improvements in machine translation quality by creating a datastore of cached examples. However, these improvements have been limited to high-resource language pairs, with …
View article: Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens
Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens Open
We argue that translation quality alone is not a sufficient metric for measuring knowledge transfer in multilingual neural machine translation. To support this claim, we introduce Representational Transfer Potential (RTP), which measures r…
View article: UvA-MT’s Participation in the WMT 2023 General Translation Shared Task
UvA-MT’s Participation in the WMT 2023 General Translation Shared Task Open
This paper describes the UvA-MT’s submission to the WMT 2023 shared task on general machine translation. We participate in the constrained track in two directions: English ↔ Hebrew. In this competition, we show that by using one model to h…
View article: ChatGPT is not a good indigenous translator
ChatGPT is not a good indigenous translator Open
This report investigates the continuous challenges of Machine Translation (MT) systems on indigenous and extremely low-resource language pairs. Despite the notable achievements of Large Language Models (LLMs) that excel in various tasks, t…
View article: Towards a general purpose machine translation system for Sranantongo
Towards a general purpose machine translation system for Sranantongo Open
Machine translation for Sranantongo (Sranan, srn), a low-resource Creole language spoken predominantly in Surinam, is virgin territory. In this study we create a general purpose machine translation system for srn. In order to facilitate th…
View article: Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks Open
How well can NLP models generalize to a variety of unseen tasks when provided with task instructions? To address this question, we first introduce Super-NaturalInstructions, a benchmark of 1,616 diverse NLP tasks and their expert-written i…
View article: Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks Open
Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Isha…
View article: Conditional Image Generation and Manipulation for User-Specified Content
Conditional Image Generation and Manipulation for User-Specified Content Open
In recent years, Generative Adversarial Networks (GANs) have improved steadily towards generating increasingly impressive real-world images. It is useful to steer the image generation process for purposes such as content creation. This can…