FedDAA: Dynamic Client Clustering for Concept Drift Adaptation in Federated Learning

TL;DR

FedDAA introduces a dynamic client clustering framework in federated learning to effectively identify and adapt to multiple sources of concept drift, significantly improving model accuracy on benchmark datasets.

Contribution

The paper presents FedDAA, a novel federated learning method that dynamically detects and adapts to different types of concept drift, addressing limitations of existing approaches.

Findings

Achieves 7.84% to 8.52% accuracy improvements over state-of-the-art methods.

Effectively distinguishes real drift from virtual and label drift.

Provides theoretical convergence guarantees.

Abstract

In federated learning (FL), the data distribution of each client may change over time, introducing both temporal and spatial data heterogeneity, known as concept drift. Data heterogeneity arises from three drift sources: real drift (a shift in the conditional distribution P(y|x)), virtual drift (a shift in the input distribution P(x)), and label drift (a shift in the label distribution P(y)). However, most existing FL methods addressing concept drift primarily focus on real drift. When clients experience virtual or label drift, these methods often fail to selectively retain useful historical knowledge, leading to catastrophic forgetting. A key challenge lies in distinguishing different sources of drift, as they require distinct adaptation strategies: real drift calls for discarding outdated data, while virtual or label drift benefits from retaining historical data. Without explicitly identifying the drift sources, a general adaptation strategy is suboptimal and may harm generalization. To address this challenge, we propose FedDAA, a dynamic clustered FL framework designed to adapt to multi-source concept drift while preserving valuable historical knowledge. Specifically, FedDAA integrates three modules: a cluster number determination module to find the optimal number of clusters; a real drift detection module to distinguish real drift from virtual/label drift; and a concept drift adaptation module to adapt to new data while retaining useful historical information. We provide theoretical convergence guarantees, and experiments show that FedDAA achieves 7.84% to 8.52% accuracy improvements over state-of-the-art methods on Fashion-MNIST, CIFAR-10, and CIFAR-100.