FLoRA: Fused forward-backward adapters for parameter efficient fine-tuning and reducing inference-time latencies of LLMs

TL;DR

FLoRA introduces fused forward-backward adapters that enhance fine-tuning accuracy and reduce inference latency of large language models by combining ideas from LoRA and parallel adapters.

Contribution

The paper proposes FLoRA, a novel fused adapter method that improves fine-tuning performance and efficiency of LLMs, addressing gaps in existing PEFT techniques.

Findings

FLoRA outperforms LoRA in accuracy on downstream tasks.

FLoRA reduces inference latency compared to traditional adapters.

FLoRA maintains similar parameter efficiency as LoRA.

Abstract

As the large language models (LLMs) grow in size each day, efficient training and fine-tuning has never been as important as nowadays. This resulted in the great interest in parameter efficient fine-tuning (PEFT), and effective methods including low-rank adapters (LoRA) has emerged. Although the various PEFT methods have been studied extensively in the recent years, the greater part of the subject remains unexplored with the huge degree of freedom. In this paper, we propose FLoRA, a family of fused forward-backward adapters (FFBA) for parameter-efficient fine-tuning of LLMs on downstream tasks. The FFBA combine ideas from the popular LoRA and parallel adapters to improve the overall fine-tuning accuracies. At the same time, latencies are minimized by fusing the forward and backward adapters into existing projection layers of the base model. Experimental results show that the proposed FFB adapters perform significantly better than the popularly used LoRA in both accuracy and latency for a similar parameter budget.