MTA: A Merge-then-Adapt Framework for Personalized Large Language Model

TL;DR

The paper introduces MTA, a scalable framework for personalized large language models that dynamically merges shared meta-LoRA modules to improve personalization without high storage costs.

Contribution

MTA proposes a novel merge-then-adapt framework with dynamic personalization, reducing storage and enhancing few-shot personalization for PLLMs.

Findings

Outperforms SOTA methods on LaMP benchmark

Supports dynamic, scalable personalization without user-specific storage

Effective in few-shot personalization scenarios

Abstract

Personalized Large Language Models (PLLMs) aim to align model outputs with individual user preferences, a crucial capability for user-centric applications. However, the prevalent approach of fine-tuning a separate module for each user faces two major limitations: (1) storage costs scale linearly with the number of users, rendering the method unscalable; and (2) fine-tuning a static model from scratch often yields suboptimal performance for users with sparse data. To address these challenges, we propose MTA, a Merge-then-Adapt framework for PLLMs. MTA comprises three key stages. First, we construct a shared Meta-LoRA Bank by selecting anchor users and pre-training meta-personalization traits within meta-LoRA modules. Second, to ensure scalability and enable dynamic personalization combination beyond static models, we introduce an Adaptive LoRA Fusion stage. This stage retrieves and dynamically merges the most relevant anchor meta-LoRAs to synthesize a user-specific one, thereby eliminating the need for user-specific storage and supporting more flexible personalization. Third, we propose a LoRA Stacking for Few-Shot Personalization stage, which applies an additional ultra-low-rank, lightweight LoRA module on top of the merged LoRA. Fine-tuning this module enables effective personalization under few-shot settings. Extensive experiments on the LaMP benchmark demonstrate that our approach outperforms existing SOTA methods across multiple tasks.