Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation

TL;DR

This paper introduces MonteCLoRA, a Bayesian reparameterization method for low-rank adaptation of large language models, improving stability, accuracy, and robustness in fine-tuning with minimal additional parameters.

Contribution

It proposes MonteCLoRA, a novel Bayesian approach that reduces variance and hyperparameter sensitivity in low-rank fine-tuning of LLMs, enhancing stability and performance.

Findings

Achieves 0.5% and 1.6% accuracy improvements on NLP tasks.

Demonstrates 50% and 62% reduction in performance spread on generative tasks.

Provides theoretical and empirical evidence of improved parameter estimation.

Abstract

Large Language Models (LLMs) are highly resource-intensive to fine-tune due to their enormous size. While low-rank adaptation is a prominent parameter-efficient fine-tuning approach, it suffers from sensitivity to hyperparameter choices, leading to instability in model performance on fine-tuning downstream tasks. This paper highlights the importance of effective parameterization in low-rank fine-tuning to reduce estimator variance and enhance the stability of final model outputs. We propose MonteCLoRA, an efficient fine-tuning technique that employs Monte Carlo estimation to learn an unbiased posterior estimation of low-rank parameters with low expected variance, stabilizing fine-tuned LLMs with only O(r) additional parameters, for a given rank r. MonteCLoRA shows 0.5% and 1.6% improvements in accuracy and robustness over unregularized low-rank adaptation method on natural language understanding tasks with pre-trained RoBERTa-base. Furthermore, in generative tasks with pre-trained LLaMA-1-7B and LLaMA-3.2-3B-Instruct, MonteCLoRA demonstrates robust performance with 50% and 62% lower spreads respectively than the contemporary efficient fine-tuning methods. The theoretical and empirical results presented in the paper underscore how parameterization and hyperpriors balance exploration-exploitation in the low-rank parametric space, therefore leading to more optimal and robust parameter estimation during efficient fine-tuning.