Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation Article Swipe

MAKE IMPACT

Ayan Sengupta , Vaibhav Seth , Ashok Kumar Pathak , Natraj Raman , Sriram Gopalakrishnan , Tanmoy Chakraborty · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2411.04358

Large Language Models (LLMs) are highly resource-intensive to fine-tune due to their enormous size. While low-rank adaptation is a prominent parameter-efficient fine-tuning approach, it suffers from sensitivity to hyperparameter choices, leading to instability in model performance on fine-tuning downstream tasks. This paper highlights the importance of effective parameterization in low-rank fine-tuning to reduce estimator variance and enhance the stability of final model outputs. We propose MonteCLoRA, an efficient fine-tuning technique that employs Monte Carlo estimation to learn an unbiased posterior estimation of low-rank parameters with low expected variance, stabilizing fine-tuned LLMs with only O(r) additional parameters, for a given rank r. MonteCLoRA shows 0.5% and 1.6% improvements in accuracy and robustness over unregularized low-rank adaptation method on natural language understanding tasks with pre-trained RoBERTa-base. Furthermore, in generative tasks with pre-trained LLaMA-1-7B and LLaMA-3.2-3B-Instruct, MonteCLoRA demonstrates robust performance with 50% and 62% lower spreads respectively than the contemporary efficient fine-tuning methods. The theoretical and empirical results presented in the paper underscore how parameterization and hyperpriors balance exploration-exploitation in the low-rank parametric space, therefore leading to more optimal and robust parameter estimation during efficient fine-tuning.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2411.04358

PDF

https://arxiv.org/pdf/2411.04358

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4404388959

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4404388959

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2411.04358

Digital Object Identifier

Title

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-11-07

Full publication date if available

Authors

Ayan Sengupta, Vaibhav Seth, Ashok Kumar Pathak, Natraj Raman, Sriram Gopalakrishnan, Tanmoy Chakraborty

List of authors in order

Landing page

https://arxiv.org/abs/2411.04358

Publisher landing page

PDF URL

https://arxiv.org/pdf/2411.04358

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2411.04358

Direct OA link when available

Concepts

Adaptation (eye), Bayesian probability, Rank (graph theory), Econometrics, Computer science, Economics, Mathematics, Artificial intelligence, Psychology, Combinatorics, Neuroscience

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload