Anke Tang
YOU?
Author Swipe
View article: Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent
Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent Open
Merging multiple expert models offers a promising approach for performing multi-task learning without accessing their original data. Existing methods attempt to alleviate task conflicts by sparsifying task vectors or promoting orthogonalit…
View article: SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models
SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models Open
Deep model training on extensive datasets is increasingly becoming cost-prohibitive, prompting the widespread adoption of deep model fusion techniques to leverage knowledge from pre-existing models. From simple weight averaging to more sop…
View article: Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion Open
Merging models fine-tuned from a common, extensively pre-trained large model but specialized for different tasks has been demonstrated as a cheap and scalable strategy to construct a multi-task model that performs well across diverse tasks…
View article: Learning from models beyond fine-tuning
Learning from models beyond fine-tuning Open
Foundation models (FM) have demonstrated remarkable performance across a wide range of tasks (especially in the fields of natural language processing and computer vision), primarily attributed to their ability to comprehend instructions an…
View article: Parameter Efficient Multi-task Model Fusion with Partial Linearization
Parameter Efficient Multi-task Model Fusion with Partial Linearization Open
Large pre-trained models have enabled significant advances in machine learning and served as foundation components. Model fusion methods, such as task arithmetic, have been proven to be powerful and scalable to incorporate fine-tuned weigh…
View article: Improving Heterogeneous Model Reuse by Density Estimation
Improving Heterogeneous Model Reuse by Density Estimation Open
This paper studies multiparty learning, aiming to learn a model using the private data of different participants. Model reuse is a promising solution for multiparty learning, assuming that a local model has been trained for each party. Con…
View article: Improving Heterogeneous Model Reuse by Density Estimation
Improving Heterogeneous Model Reuse by Density Estimation Open
This paper studies multiparty learning, aiming to learn a model using the private data of different participants. Model reuse is a promising solution for multiparty learning, assuming that a local model has been trained for each party. Con…