Personalized Collaborative Fine-Tuning for On-Device Large Language Models

TL;DR

This paper introduces a novel on-device collaborative fine-tuning method for large language models that leverages trust-weighted gradient aggregation and low-rank adaptation to improve performance with limited local data.

Contribution

It proposes three trust-based gradient aggregation schemes and integrates LoRA to reduce communication, outperforming existing methods in diverse data scenarios.

Findings

Outperforms FedAvg and local fine-tuning methods

Effective in heterogeneous and scarce data environments

Reduces communication overhead with LoRA

Abstract

We explore on-device self-supervised collaborative fine-tuning of large language models with limited local data availability. Taking inspiration from the collaborative learning community, we introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based. To minimize communication overhead, we integrate Low-Rank Adaptation (LoRA) and only exchange LoRA weight updates. Our protocols, driven by prediction and performance metrics, surpass both FedAvg and local fine-tuning methods, which is particularly evident in realistic scenarios with more diverse local data distributions. The results underscore the effectiveness of our approach in addressing heterogeneity and scarcity within local datasets.