The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation Article Swipe

PDF

YOU? · · 2025 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2505.13090

Prior research diverges on language diversity in LLM fine-tuning: Some studies report benefits while others find no advantages. Through controlled fine-tuning experiments across 132 translation directions, we systematically resolve these disparities. We find that expanding language diversity during fine-tuning improves translation quality for both unsupervised and -- surprisingly -- supervised pairs, despite less diverse models being fine-tuned exclusively on these supervised pairs. However, benefits plateau or decrease beyond a certain diversity threshold. We show that increased language diversity creates more language-agnostic representations. These representational adaptations help explain the improved performance in models fine-tuned with greater diversity.

Related Topics

When Life Gives You Tangerines

Being Funny In A Foreign Language

Claude (Language Model)

Language Family

Proto-Indo-European Language

Swahili Language

Rust (Programming Language)

Scratch (Programming Language)

Scots Language

Maltese Language

Vietnamese Language

Large Hadron Collider

Concepts

No concepts available.

Metadata

Type: preprint
Language: en
Landing Page: http://arxiv.org/abs/2505.13090
PDF: https://arxiv.org/pdf/2505.13090
OA Status: green
OpenAlex ID: https://openalex.org/W4417287168

All OpenAlex metadata

Raw OpenAlex JSON

OpenAlex ID: https://openalex.org/W4417287168

Canonical identifier for this work in OpenAlex
DOI: https://doi.org/10.48550/arxiv.2505.13090

Digital Object Identifier
Title: The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation

Work title
Type: preprint

OpenAlex work type
Language: en

Primary language
Publication year: 2025

Year of publication
Publication date: 2025-05-19

Full publication date if available
Authors: David Stap, Christof Monz

List of authors in order
Landing page: https://arxiv.org/abs/2505.13090

Publisher landing page
PDF URL: https://arxiv.org/pdf/2505.13090

Direct link to full text PDF
Open access: Yes

Whether a free full text is available
OA status: green

Open access status per OpenAlex
OA URL: https://arxiv.org/pdf/2505.13090

Direct OA link when available
Cited by: 0

Total citation count in OpenAlex

Full payload

id	https://openalex.org/W4417287168
doi	https://doi.org/10.48550/arxiv.2505.13090
ids.doi	https://doi.org/10.48550/arxiv.2505.13090
ids.openalex	https://openalex.org/W4417287168
fwci
type	preprint
title	The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation
biblio.issue
biblio.volume
biblio.last_page
biblio.first_page
is_xpac	False
apc_list
apc_paid
language	en
locations[0].id	pmh:oai:arXiv.org:2505.13090
locations[0].is_oa	True
locations[0].source.id	https://openalex.org/S4306400194
locations[0].source.issn
locations[0].source.type	repository
locations[0].source.is_oa	True
locations[0].source.issn_l
locations[0].source.is_core	False
locations[0].source.is_in_doaj	False
locations[0].source.display_name	arXiv (Cornell University)
locations[0].source.host_organization	https://openalex.org/I205783295
locations[0].source.host_organization_name	Cornell University
locations[0].source.host_organization_lineage	https://openalex.org/I205783295
locations[0].license	cc-by
locations[0].pdf_url	https://arxiv.org/pdf/2505.13090
locations[0].version	submittedVersion
locations[0].raw_type	text
locations[0].license_id	https://openalex.org/licenses/cc-by
locations[0].is_accepted	False
locations[0].is_published	False
locations[0].raw_source_name
locations[0].landing_page_url	http://arxiv.org/abs/2505.13090
locations[1].id	doi:10.48550/arxiv.2505.13090
locations[1].is_oa	True
locations[1].source.id	https://openalex.org/S4306400194
locations[1].source.issn
locations[1].source.type	repository
locations[1].source.is_oa	True
locations[1].source.issn_l
locations[1].source.is_core	False
locations[1].source.is_in_doaj	False
locations[1].source.display_name	arXiv (Cornell University)
locations[1].source.host_organization	https://openalex.org/I205783295
locations[1].source.host_organization_name	Cornell University
locations[1].source.host_organization_lineage	https://openalex.org/I205783295
locations[1].license	cc-by
locations[1].pdf_url
locations[1].version
locations[1].raw_type	article
locations[1].license_id	https://openalex.org/licenses/cc-by
locations[1].is_accepted	False
locations[1].is_published
locations[1].raw_source_name
locations[1].landing_page_url	https://doi.org/10.48550/arxiv.2505.13090
indexed_in	arxiv, datacite
authorships[0].author.id	https://openalex.org/A5015155057
authorships[0].author.orcid
authorships[0].author.display_name	David Stap
authorships[0].author_position	first
authorships[0].raw_author_name	Stap, David
authorships[0].is_corresponding	False
authorships[1].author.id	https://openalex.org/A5109059955
authorships[1].author.orcid
authorships[1].author.display_name	Christof Monz
authorships[1].author_position	last
authorships[1].raw_author_name	Monz, Christof
authorships[1].is_corresponding	False
has_content.pdf	True
has_content.grobid_xml	True
is_paratext	False
open_access.is_oa	True
open_access.oa_url	https://arxiv.org/pdf/2505.13090
open_access.oa_status	green
open_access.any_repository_has_fulltext	False
created_date	2025-10-10T00:00:00
display_name	The Effect of Language Diversity When Fine-Tuning Large Language Models for Translation
has_fulltext	True
is_retracted	False
updated_date	2025-12-13T09:19:10.953794
primary_topic
cited_by_count	0
locations_count	2
best_oa_location.id	pmh:oai:arXiv.org:2505.13090
best_oa_location.is_oa	True
best_oa_location.source.id	https://openalex.org/S4306400194
best_oa_location.source.issn
best_oa_location.source.type	repository
best_oa_location.source.is_oa	True
best_oa_location.source.issn_l
best_oa_location.source.is_core	False
best_oa_location.source.is_in_doaj	False
best_oa_location.source.display_name	arXiv (Cornell University)
best_oa_location.source.host_organization	https://openalex.org/I205783295
best_oa_location.source.host_organization_name	Cornell University
best_oa_location.source.host_organization_lineage	https://openalex.org/I205783295
best_oa_location.license	cc-by
best_oa_location.pdf_url	https://arxiv.org/pdf/2505.13090
best_oa_location.version	submittedVersion
best_oa_location.raw_type	text
best_oa_location.license_id	https://openalex.org/licenses/cc-by
best_oa_location.is_accepted	False
best_oa_location.is_published	False
best_oa_location.raw_source_name
best_oa_location.landing_page_url	http://arxiv.org/abs/2505.13090
primary_location.id	pmh:oai:arXiv.org:2505.13090
primary_location.is_oa	True
primary_location.source.id	https://openalex.org/S4306400194
primary_location.source.issn
primary_location.source.type	repository
primary_location.source.is_oa	True
primary_location.source.issn_l
primary_location.source.is_core	False
primary_location.source.is_in_doaj	False
primary_location.source.display_name	arXiv (Cornell University)
primary_location.source.host_organization	https://openalex.org/I205783295
primary_location.source.host_organization_name	Cornell University
primary_location.source.host_organization_lineage	https://openalex.org/I205783295
primary_location.license	cc-by
primary_location.pdf_url	https://arxiv.org/pdf/2505.13090
primary_location.version	submittedVersion
primary_location.raw_type	text
primary_location.license_id	https://openalex.org/licenses/cc-by
primary_location.is_accepted	False
primary_location.is_published	False
primary_location.raw_source_name
primary_location.landing_page_url	http://arxiv.org/abs/2505.13090
publication_date	2025-05-19
publication_year	2025
referenced_works_count	0
abstract_inverted_index.a	68
abstract_inverted_index.--	46, 48
abstract_inverted_index.We	31, 72
abstract_inverted_index.in	6, 90
abstract_inverted_index.no	16
abstract_inverted_index.on	3, 58
abstract_inverted_index.or	65
abstract_inverted_index.we	26
abstract_inverted_index.132	23
abstract_inverted_index.LLM	7
abstract_inverted_index.and	45
abstract_inverted_index.for	42
abstract_inverted_index.the	87
abstract_inverted_index.Some	9
abstract_inverted_index.both	43
abstract_inverted_index.find	15, 32
abstract_inverted_index.help	85
abstract_inverted_index.less	52
abstract_inverted_index.more	79
abstract_inverted_index.show	73
abstract_inverted_index.that	33, 74
abstract_inverted_index.with	93
abstract_inverted_index.Prior	0
abstract_inverted_index.These	82
abstract_inverted_index.being	55
abstract_inverted_index.these	29, 59
abstract_inverted_index.while	13
abstract_inverted_index.across	22
abstract_inverted_index.beyond	67
abstract_inverted_index.during	37
abstract_inverted_index.models	54, 91
abstract_inverted_index.others	14
abstract_inverted_index.pairs,	50
abstract_inverted_index.pairs.	61
abstract_inverted_index.report	11
abstract_inverted_index.Through	18
abstract_inverted_index.certain	69
abstract_inverted_index.creates	78
abstract_inverted_index.despite	51
abstract_inverted_index.diverse	53
abstract_inverted_index.explain	86
abstract_inverted_index.greater	94
abstract_inverted_index.plateau	64
abstract_inverted_index.quality	41
abstract_inverted_index.resolve	28
abstract_inverted_index.studies	10
abstract_inverted_index.However,	62
abstract_inverted_index.benefits	12, 63
abstract_inverted_index.decrease	66
abstract_inverted_index.diverges	2
abstract_inverted_index.improved	88
abstract_inverted_index.improves	39
abstract_inverted_index.language	4, 35, 76
abstract_inverted_index.research	1
abstract_inverted_index.diversity	5, 36, 70, 77
abstract_inverted_index.expanding	34
abstract_inverted_index.increased	75
abstract_inverted_index.controlled	19
abstract_inverted_index.diversity.	95
abstract_inverted_index.fine-tuned	56, 92
abstract_inverted_index.supervised	49, 60
abstract_inverted_index.threshold.	71
abstract_inverted_index.adaptations	84
abstract_inverted_index.advantages.	17
abstract_inverted_index.directions,	25
abstract_inverted_index.exclusively	57
abstract_inverted_index.experiments	21
abstract_inverted_index.fine-tuning	20, 38
abstract_inverted_index.performance	89
abstract_inverted_index.translation	24, 40
abstract_inverted_index.disparities.	30
abstract_inverted_index.fine-tuning:	8
abstract_inverted_index.surprisingly	47
abstract_inverted_index.unsupervised	44
abstract_inverted_index.systematically	27
abstract_inverted_index.representational	83
abstract_inverted_index.representations.	81
abstract_inverted_index.language-agnostic	80
cited_by_percentile_year
countries_distinct_count	0
institutions_distinct_count	2
citation_normalized_percentile