Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models

Exploring foci of: arXiv (Cornell University) Data Mixing Optimization for Supervised Fine-Tuning of Large Language Models August 2025 • Yuan Li, Zhengzhong Liu, Eric P. Xing Optimizing data mixtures for supervised fine-tuning (SFT) of large language models (LLMs) is critical for developing general-purpose models, yet this area remains underexplored. In this paper, we frame data mixing as an optimization problem and introduce a novel method designed to minimize validation loss. Our approach parametrizes the loss by modeling effective data transferred and leveraging scaling laws for fine-tuning. By experimenting with various small-scale data mixtures, we fit these parameters and derive … Open Article Page

Ant Colony Optimization Algorithms General Data Protection Regulation Live Sound Mixing Search Engine Optimization Large Hadron Collider Extremely Large Telescope Data Science Jerry & Marge Go Large Data Open Article

Data Center Large Intestine Big Data Data Analysis Data Mining Ntt Data Very Large Telescope Computer Data Storage Atacama Large Millimeter Array Open Article

Data Warehouse Data Structure Training, Validation, And Test Data Sets List Of U.S. Cities With Large Hispanic Populations Particle Swarm Optimization Little And Large Open Article