Look Back for More: Harnessing Historical Sequential Updates for Personalized Federated Adapter Tuning

TL;DR

This paper introduces pFedSeq, a novel personalized federated learning framework that leverages historical client updates using a sequential learner with SSM architecture to improve model personalization.

Contribution

The paper proposes pFedSeq, which utilizes historical update sequences and a state space model to enhance personalized model tuning in federated learning.

Findings

pFedSeq outperforms existing PFL methods on benchmark datasets.

Sequential modeling of updates improves personalization accuracy.

Using SSM captures complex update relations effectively.

Abstract

Personalized federated learning (PFL) studies effective model personalization to address the data heterogeneity issue among clients in traditional federated learning (FL). Existing PFL approaches mainly generate personalized models by relying solely on the clients' latest updated models while ignoring their previous updates, which may result in suboptimal personalized model learning. To bridge this gap, we propose a novel framework termed pFedSeq, designed for personalizing adapters to fine-tune a foundation model in FL. In pFedSeq, the server maintains and trains a sequential learner, which processes a sequence of past adapter updates from clients and generates calibrations for personalized adapters. To effectively capture the cross-client and cross-step relations hidden in previous updates and generate high-performing personalized adapters, pFedSeq adopts the powerful selective state space model (SSM) as the architecture of sequential learner. Through extensive experiments on four public benchmark datasets, we demonstrate the superiority of pFedSeq over state-of-the-art PFL methods.