Personalized Federated Learning on Data with Dynamic Heterogeneity under Limited Storage

TL;DR

This paper introduces pFedGRP, a personalized federated learning framework that uses a category-decoupled generative replay architecture to address data heterogeneity and storage limitations, improving local model personalization.

Contribution

The paper proposes a novel pFL method with a category-decoupled generator and local data reconstruction to mitigate catastrophic forgetting and enhance personalization under data heterogeneity.

Findings

pFedGRP outperforms eight baseline methods across five datasets.

The approach effectively reduces data generation and storage costs.

It improves local model accuracy and personalization.

Abstract

Recently, a large number of data sources opened up by informatization intensify the data heterogeneity, the faster speed of data generation and the gradual implementation of data regulations limit the storage time of data. In personalized Federated Learning (pFL), clients train customized models to meet their personal objectives. However, due to the time-varying local data heterogeneity and the inaccessibility of previous data, existing pFL methods not only fail to solve the catastrophic forgetting of local models, but also difficult to estimate the degree of collaboration between clients. To address this issue, our core idea is a low consumption and high-quality generative replay architecture. Specifically, we decouple the generator by category to reduce the generation error of each category while mitigating catastrophic forgetting, use local model to improving the quality of generated data and reducing the update frequency of generator, and propose a local data reconstruction scheme to reduce data generation while adjusting the proportion of data categories. Based on above, we propose our pFL framework, pFedGRP, to achieve personalized aggregation and local knowledge transfer. Comprehensive experiments on five datasets with multiple settings show the superiority of pFedGRP over eight baseline methods.