CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning Article Swipe
Yangfan Ye
,
Xiaocheng Feng
,
Z. Y. Yuan
,
Xiachong Feng
,
L. Q. Qin
,
Lei Huang
,
Weitao Ma
,
Yudong Huang
,
Zhirui Zhang
,
Yunfei Lu
,
Xiaohui Yan
,
Duyu Tang
,
Dandan Tu
,
Bing Qin
·
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2506.00875
YOU?
·
· 2025
· Open Access
·
· DOI: https://doi.org/10.48550/arxiv.2506.00875
Current large language models (LLMs) often exhibit imbalanced multilingual capabilities due to their English-centric training corpora. To address this, existing fine-tuning approaches operating at the data-level (e.g., through data augmentation or distillation) typically introduce implicit cross-lingual alignment, overlooking the potential for more profound, latent-level cross-lingual interactions. In this work, we propose CC-Tuning, a novel multilingual fine-tuning paradigm that explicitly establishes a cross-lingual connection mechanism at the latent level. During training, CC-Tuning fuses the feed forward activations from both English and non-English inputs, enabling the model to benefit from both linguistic resources. This process is facilitated with a trainable Decision Maker that identifies beneficial activations. Furthermore, during inference, a Transform Matrix is utilized to simulate the cross-lingual connection under monolingual setting through representation transformation. Our experiments on six benchmarks covering 22 languages show that CC-Tuning outperforms vanilla SFT and offers a strong latent-level alternative to data-level augmentation methods. Further analysis also highlights the practicality of CC-Tuning and the potential of latent-level cross-lingual interactions in advancing the multilingual performance of LLMs.
Related Topics
Concepts
No concepts available.
Metadata
- Type
- preprint
- Language
- en
- Landing Page
- http://arxiv.org/abs/2506.00875
- https://arxiv.org/pdf/2506.00875
- OA Status
- green
- OpenAlex ID
- https://openalex.org/W4414893766
All OpenAlex metadata
Raw OpenAlex JSON
- OpenAlex ID
-
https://openalex.org/W4414893766Canonical identifier for this work in OpenAlex
- DOI
-
https://doi.org/10.48550/arxiv.2506.00875Digital Object Identifier
- Title
-
CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-TuningWork title
- Type
-
preprintOpenAlex work type
- Language
-
enPrimary language
- Publication year
-
2025Year of publication
- Publication date
-
2025-06-01Full publication date if available
- Authors
-
Yangfan Ye, Xiaocheng Feng, Z. Y. Yuan, Xiachong Feng, L. Q. Qin, Lei Huang, Weitao Ma, Yudong Huang, Zhirui Zhang, Yunfei Lu, Xiaohui Yan, Duyu Tang, Dandan Tu, Bing QinList of authors in order
- Landing page
-
https://arxiv.org/abs/2506.00875Publisher landing page
- PDF URL
-
https://arxiv.org/pdf/2506.00875Direct link to full text PDF
- Open access
-
YesWhether a free full text is available
- OA status
-
greenOpen access status per OpenAlex
- OA URL
-
https://arxiv.org/pdf/2506.00875Direct OA link when available
- Cited by
-
0Total citation count in OpenAlex
Full payload
| id | https://openalex.org/W4414893766 |
|---|---|
| doi | https://doi.org/10.48550/arxiv.2506.00875 |
| ids.doi | https://doi.org/10.48550/arxiv.2506.00875 |
| ids.openalex | https://openalex.org/W4414893766 |
| fwci | |
| type | preprint |
| title | CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning |
| biblio.issue | |
| biblio.volume | |
| biblio.last_page | |
| biblio.first_page | |
| topics[0].id | https://openalex.org/T12031 |
| topics[0].field.id | https://openalex.org/fields/17 |
| topics[0].field.display_name | Computer Science |
| topics[0].score | 0.8245000243186951 |
| topics[0].domain.id | https://openalex.org/domains/3 |
| topics[0].domain.display_name | Physical Sciences |
| topics[0].subfield.id | https://openalex.org/subfields/1702 |
| topics[0].subfield.display_name | Artificial Intelligence |
| topics[0].display_name | Speech and dialogue systems |
| topics[1].id | https://openalex.org/T10403 |
| topics[1].field.id | https://openalex.org/fields/32 |
| topics[1].field.display_name | Psychology |
| topics[1].score | 0.8027999997138977 |
| topics[1].domain.id | https://openalex.org/domains/2 |
| topics[1].domain.display_name | Social Sciences |
| topics[1].subfield.id | https://openalex.org/subfields/3205 |
| topics[1].subfield.display_name | Experimental and Cognitive Psychology |
| topics[1].display_name | Phonetics and Phonology Research |
| topics[2].id | https://openalex.org/T10181 |
| topics[2].field.id | https://openalex.org/fields/17 |
| topics[2].field.display_name | Computer Science |
| topics[2].score | 0.7059999704360962 |
| topics[2].domain.id | https://openalex.org/domains/3 |
| topics[2].domain.display_name | Physical Sciences |
| topics[2].subfield.id | https://openalex.org/subfields/1702 |
| topics[2].subfield.display_name | Artificial Intelligence |
| topics[2].display_name | Natural Language Processing Techniques |
| is_xpac | False |
| apc_list | |
| apc_paid | |
| language | en |
| locations[0].id | pmh:oai:arXiv.org:2506.00875 |
| locations[0].is_oa | True |
| locations[0].source.id | https://openalex.org/S4306400194 |
| locations[0].source.issn | |
| locations[0].source.type | repository |
| locations[0].source.is_oa | True |
| locations[0].source.issn_l | |
| locations[0].source.is_core | False |
| locations[0].source.is_in_doaj | False |
| locations[0].source.display_name | arXiv (Cornell University) |
| locations[0].source.host_organization | https://openalex.org/I205783295 |
| locations[0].source.host_organization_name | Cornell University |
| locations[0].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[0].license | |
| locations[0].pdf_url | https://arxiv.org/pdf/2506.00875 |
| locations[0].version | submittedVersion |
| locations[0].raw_type | text |
| locations[0].license_id | |
| locations[0].is_accepted | False |
| locations[0].is_published | False |
| locations[0].raw_source_name | |
| locations[0].landing_page_url | http://arxiv.org/abs/2506.00875 |
| locations[1].id | doi:10.48550/arxiv.2506.00875 |
| locations[1].is_oa | True |
| locations[1].source.id | https://openalex.org/S4306400194 |
| locations[1].source.issn | |
| locations[1].source.type | repository |
| locations[1].source.is_oa | True |
| locations[1].source.issn_l | |
| locations[1].source.is_core | False |
| locations[1].source.is_in_doaj | False |
| locations[1].source.display_name | arXiv (Cornell University) |
| locations[1].source.host_organization | https://openalex.org/I205783295 |
| locations[1].source.host_organization_name | Cornell University |
| locations[1].source.host_organization_lineage | https://openalex.org/I205783295 |
| locations[1].license | |
| locations[1].pdf_url | |
| locations[1].version | |
| locations[1].raw_type | article |
| locations[1].license_id | |
| locations[1].is_accepted | False |
| locations[1].is_published | |
| locations[1].raw_source_name | |
| locations[1].landing_page_url | https://doi.org/10.48550/arxiv.2506.00875 |
| indexed_in | arxiv, datacite |
| authorships[0].author.id | https://openalex.org/A5110972941 |
| authorships[0].author.orcid | |
| authorships[0].author.display_name | Yangfan Ye |
| authorships[0].author_position | first |
| authorships[0].raw_author_name | Ye, Yangfan |
| authorships[0].is_corresponding | False |
| authorships[1].author.id | https://openalex.org/A5073213620 |
| authorships[1].author.orcid | https://orcid.org/0000-0001-6011-0496 |
| authorships[1].author.display_name | Xiaocheng Feng |
| authorships[1].author_position | middle |
| authorships[1].raw_author_name | Feng, Xiaocheng |
| authorships[1].is_corresponding | False |
| authorships[2].author.id | https://openalex.org/A5076514259 |
| authorships[2].author.orcid | https://orcid.org/0009-0007-5102-8504 |
| authorships[2].author.display_name | Z. Y. Yuan |
| authorships[2].author_position | middle |
| authorships[2].raw_author_name | Yuan, Zekun |
| authorships[2].is_corresponding | False |
| authorships[3].author.id | https://openalex.org/A5066977101 |
| authorships[3].author.orcid | https://orcid.org/0000-0002-4761-7484 |
| authorships[3].author.display_name | Xiachong Feng |
| authorships[3].author_position | middle |
| authorships[3].raw_author_name | Feng, Xiachong |
| authorships[3].is_corresponding | False |
| authorships[4].author.id | https://openalex.org/A5092102582 |
| authorships[4].author.orcid | https://orcid.org/0009-0000-6141-3931 |
| authorships[4].author.display_name | L. Q. Qin |
| authorships[4].author_position | middle |
| authorships[4].raw_author_name | Qin, Libo |
| authorships[4].is_corresponding | False |
| authorships[5].author.id | https://openalex.org/A5102875046 |
| authorships[5].author.orcid | https://orcid.org/0009-0001-5756-0215 |
| authorships[5].author.display_name | Lei Huang |
| authorships[5].author_position | middle |
| authorships[5].raw_author_name | Huang, Lei |
| authorships[5].is_corresponding | False |
| authorships[6].author.id | https://openalex.org/A5111073904 |
| authorships[6].author.orcid | https://orcid.org/0009-0007-8631-3858 |
| authorships[6].author.display_name | Weitao Ma |
| authorships[6].author_position | middle |
| authorships[6].raw_author_name | Ma, Weitao |
| authorships[6].is_corresponding | False |
| authorships[7].author.id | https://openalex.org/A5100768893 |
| authorships[7].author.orcid | https://orcid.org/0000-0002-9998-557X |
| authorships[7].author.display_name | Yudong Huang |
| authorships[7].author_position | middle |
| authorships[7].raw_author_name | Huang, Yichong |
| authorships[7].is_corresponding | False |
| authorships[8].author.id | https://openalex.org/A5053154355 |
| authorships[8].author.orcid | |
| authorships[8].author.display_name | Zhirui Zhang |
| authorships[8].author_position | middle |
| authorships[8].raw_author_name | Zhang, Zhirui |
| authorships[8].is_corresponding | False |
| authorships[9].author.id | https://openalex.org/A5113627163 |
| authorships[9].author.orcid | https://orcid.org/0000-0001-5363-6836 |
| authorships[9].author.display_name | Yunfei Lu |
| authorships[9].author_position | middle |
| authorships[9].raw_author_name | Lu, Yunfei |
| authorships[9].is_corresponding | False |
| authorships[10].author.id | https://openalex.org/A5101427643 |
| authorships[10].author.orcid | https://orcid.org/0000-0003-0795-9321 |
| authorships[10].author.display_name | Xiaohui Yan |
| authorships[10].author_position | middle |
| authorships[10].raw_author_name | Yan, Xiaohui |
| authorships[10].is_corresponding | False |
| authorships[11].author.id | https://openalex.org/A5110628758 |
| authorships[11].author.orcid | |
| authorships[11].author.display_name | Duyu Tang |
| authorships[11].author_position | middle |
| authorships[11].raw_author_name | Tang, Duyu |
| authorships[11].is_corresponding | False |
| authorships[12].author.id | https://openalex.org/A5036303824 |
| authorships[12].author.orcid | https://orcid.org/0000-0003-3827-6486 |
| authorships[12].author.display_name | Dandan Tu |
| authorships[12].author_position | middle |
| authorships[12].raw_author_name | Tu, Dandan |
| authorships[12].is_corresponding | False |
| authorships[13].author.id | https://openalex.org/A5017671620 |
| authorships[13].author.orcid | https://orcid.org/0000-0002-2543-5604 |
| authorships[13].author.display_name | Bing Qin |
| authorships[13].author_position | last |
| authorships[13].raw_author_name | Qin, Bing |
| authorships[13].is_corresponding | False |
| has_content.pdf | False |
| has_content.grobid_xml | False |
| is_paratext | False |
| open_access.is_oa | True |
| open_access.oa_url | https://arxiv.org/pdf/2506.00875 |
| open_access.oa_status | green |
| open_access.any_repository_has_fulltext | False |
| created_date | 2025-10-10T00:00:00 |
| display_name | CC-Tuning: A Cross-Lingual Connection Mechanism for Improving Joint Multilingual Supervised Fine-Tuning |
| has_fulltext | False |
| is_retracted | False |
| updated_date | 2025-11-06T06:51:31.235846 |
| primary_topic.id | https://openalex.org/T12031 |
| primary_topic.field.id | https://openalex.org/fields/17 |
| primary_topic.field.display_name | Computer Science |
| primary_topic.score | 0.8245000243186951 |
| primary_topic.domain.id | https://openalex.org/domains/3 |
| primary_topic.domain.display_name | Physical Sciences |
| primary_topic.subfield.id | https://openalex.org/subfields/1702 |
| primary_topic.subfield.display_name | Artificial Intelligence |
| primary_topic.display_name | Speech and dialogue systems |
| cited_by_count | 0 |
| locations_count | 2 |
| best_oa_location.id | pmh:oai:arXiv.org:2506.00875 |
| best_oa_location.is_oa | True |
| best_oa_location.source.id | https://openalex.org/S4306400194 |
| best_oa_location.source.issn | |
| best_oa_location.source.type | repository |
| best_oa_location.source.is_oa | True |
| best_oa_location.source.issn_l | |
| best_oa_location.source.is_core | False |
| best_oa_location.source.is_in_doaj | False |
| best_oa_location.source.display_name | arXiv (Cornell University) |
| best_oa_location.source.host_organization | https://openalex.org/I205783295 |
| best_oa_location.source.host_organization_name | Cornell University |
| best_oa_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| best_oa_location.license | |
| best_oa_location.pdf_url | https://arxiv.org/pdf/2506.00875 |
| best_oa_location.version | submittedVersion |
| best_oa_location.raw_type | text |
| best_oa_location.license_id | |
| best_oa_location.is_accepted | False |
| best_oa_location.is_published | False |
| best_oa_location.raw_source_name | |
| best_oa_location.landing_page_url | http://arxiv.org/abs/2506.00875 |
| primary_location.id | pmh:oai:arXiv.org:2506.00875 |
| primary_location.is_oa | True |
| primary_location.source.id | https://openalex.org/S4306400194 |
| primary_location.source.issn | |
| primary_location.source.type | repository |
| primary_location.source.is_oa | True |
| primary_location.source.issn_l | |
| primary_location.source.is_core | False |
| primary_location.source.is_in_doaj | False |
| primary_location.source.display_name | arXiv (Cornell University) |
| primary_location.source.host_organization | https://openalex.org/I205783295 |
| primary_location.source.host_organization_name | Cornell University |
| primary_location.source.host_organization_lineage | https://openalex.org/I205783295 |
| primary_location.license | |
| primary_location.pdf_url | https://arxiv.org/pdf/2506.00875 |
| primary_location.version | submittedVersion |
| primary_location.raw_type | text |
| primary_location.license_id | |
| primary_location.is_accepted | False |
| primary_location.is_published | False |
| primary_location.raw_source_name | |
| primary_location.landing_page_url | http://arxiv.org/abs/2506.00875 |
| publication_date | 2025-06-01 |
| publication_year | 2025 |
| referenced_works_count | 0 |
| abstract_inverted_index.a | 52, 60, 96, 107, 139 |
| abstract_inverted_index.22 | 129 |
| abstract_inverted_index.In | 46 |
| abstract_inverted_index.To | 16 |
| abstract_inverted_index.at | 23, 64 |
| abstract_inverted_index.in | 162 |
| abstract_inverted_index.is | 93, 110 |
| abstract_inverted_index.of | 153, 158, 167 |
| abstract_inverted_index.on | 125 |
| abstract_inverted_index.or | 30 |
| abstract_inverted_index.to | 11, 85, 112, 143 |
| abstract_inverted_index.we | 49 |
| abstract_inverted_index.Our | 123 |
| abstract_inverted_index.SFT | 136 |
| abstract_inverted_index.and | 79, 137, 155 |
| abstract_inverted_index.due | 10 |
| abstract_inverted_index.for | 40 |
| abstract_inverted_index.six | 126 |
| abstract_inverted_index.the | 24, 38, 65, 72, 83, 114, 151, 156, 164 |
| abstract_inverted_index.This | 91 |
| abstract_inverted_index.also | 149 |
| abstract_inverted_index.both | 77, 88 |
| abstract_inverted_index.data | 28 |
| abstract_inverted_index.feed | 73 |
| abstract_inverted_index.from | 76, 87 |
| abstract_inverted_index.more | 41 |
| abstract_inverted_index.show | 131 |
| abstract_inverted_index.that | 57, 100, 132 |
| abstract_inverted_index.this | 47 |
| abstract_inverted_index.with | 95 |
| abstract_inverted_index.LLMs. | 168 |
| abstract_inverted_index.Maker | 99 |
| abstract_inverted_index.fuses | 71 |
| abstract_inverted_index.large | 1 |
| abstract_inverted_index.model | 84 |
| abstract_inverted_index.novel | 53 |
| abstract_inverted_index.often | 5 |
| abstract_inverted_index.their | 12 |
| abstract_inverted_index.this, | 18 |
| abstract_inverted_index.under | 117 |
| abstract_inverted_index.work, | 48 |
| abstract_inverted_index.(LLMs) | 4 |
| abstract_inverted_index.(e.g., | 26 |
| abstract_inverted_index.During | 68 |
| abstract_inverted_index.Matrix | 109 |
| abstract_inverted_index.during | 105 |
| abstract_inverted_index.latent | 66 |
| abstract_inverted_index.level. | 67 |
| abstract_inverted_index.models | 3 |
| abstract_inverted_index.offers | 138 |
| abstract_inverted_index.strong | 140 |
| abstract_inverted_index.Current | 0 |
| abstract_inverted_index.English | 78 |
| abstract_inverted_index.Further | 147 |
| abstract_inverted_index.address | 17 |
| abstract_inverted_index.benefit | 86 |
| abstract_inverted_index.exhibit | 6 |
| abstract_inverted_index.forward | 74 |
| abstract_inverted_index.inputs, | 81 |
| abstract_inverted_index.process | 92 |
| abstract_inverted_index.propose | 50 |
| abstract_inverted_index.setting | 119 |
| abstract_inverted_index.through | 27, 120 |
| abstract_inverted_index.vanilla | 135 |
| abstract_inverted_index.Decision | 98 |
| abstract_inverted_index.analysis | 148 |
| abstract_inverted_index.corpora. | 15 |
| abstract_inverted_index.covering | 128 |
| abstract_inverted_index.enabling | 82 |
| abstract_inverted_index.existing | 19 |
| abstract_inverted_index.implicit | 34 |
| abstract_inverted_index.language | 2 |
| abstract_inverted_index.methods. | 146 |
| abstract_inverted_index.paradigm | 56 |
| abstract_inverted_index.simulate | 113 |
| abstract_inverted_index.training | 14 |
| abstract_inverted_index.utilized | 111 |
| abstract_inverted_index.CC-Tuning | 70, 133, 154 |
| abstract_inverted_index.Transform | 108 |
| abstract_inverted_index.advancing | 163 |
| abstract_inverted_index.introduce | 33 |
| abstract_inverted_index.languages | 130 |
| abstract_inverted_index.mechanism | 63 |
| abstract_inverted_index.operating | 22 |
| abstract_inverted_index.potential | 39, 157 |
| abstract_inverted_index.profound, | 42 |
| abstract_inverted_index.trainable | 97 |
| abstract_inverted_index.training, | 69 |
| abstract_inverted_index.typically | 32 |
| abstract_inverted_index.CC-Tuning, | 51 |
| abstract_inverted_index.alignment, | 36 |
| abstract_inverted_index.approaches | 21 |
| abstract_inverted_index.benchmarks | 127 |
| abstract_inverted_index.beneficial | 102 |
| abstract_inverted_index.connection | 62, 116 |
| abstract_inverted_index.data-level | 25, 144 |
| abstract_inverted_index.explicitly | 58 |
| abstract_inverted_index.highlights | 150 |
| abstract_inverted_index.identifies | 101 |
| abstract_inverted_index.imbalanced | 7 |
| abstract_inverted_index.inference, | 106 |
| abstract_inverted_index.linguistic | 89 |
| abstract_inverted_index.resources. | 90 |
| abstract_inverted_index.activations | 75 |
| abstract_inverted_index.alternative | 142 |
| abstract_inverted_index.establishes | 59 |
| abstract_inverted_index.experiments | 124 |
| abstract_inverted_index.facilitated | 94 |
| abstract_inverted_index.fine-tuning | 20, 55 |
| abstract_inverted_index.monolingual | 118 |
| abstract_inverted_index.non-English | 80 |
| abstract_inverted_index.outperforms | 134 |
| abstract_inverted_index.overlooking | 37 |
| abstract_inverted_index.performance | 166 |
| abstract_inverted_index.Furthermore, | 104 |
| abstract_inverted_index.activations. | 103 |
| abstract_inverted_index.augmentation | 29, 145 |
| abstract_inverted_index.capabilities | 9 |
| abstract_inverted_index.interactions | 161 |
| abstract_inverted_index.latent-level | 43, 141, 159 |
| abstract_inverted_index.multilingual | 8, 54, 165 |
| abstract_inverted_index.practicality | 152 |
| abstract_inverted_index.cross-lingual | 35, 44, 61, 115, 160 |
| abstract_inverted_index.distillation) | 31 |
| abstract_inverted_index.interactions. | 45 |
| abstract_inverted_index.representation | 121 |
| abstract_inverted_index.English-centric | 13 |
| abstract_inverted_index.transformation. | 122 |
| cited_by_percentile_year | |
| countries_distinct_count | 0 |
| institutions_distinct_count | 14 |
| citation_normalized_percentile |