Handling Spatial-Temporal Data Heterogeneity for Federated Continual Learning via Tail Anchor

TL;DR

This paper introduces FedTA, a novel federated continual learning method that addresses spatial-temporal data heterogeneity by adjusting feature space positions, significantly reducing forgetting and improving model performance.

Contribution

The paper proposes FedTA, which uses a tail anchor mechanism and three novel components to mitigate catastrophic forgetting caused by data heterogeneity in federated continual learning.

Findings

FedTA outperforms existing FCL methods in experiments.

FedTA effectively preserves feature space structure.

Input Enhancement improves pre-trained model performance.

Abstract

Federated continual learning (FCL) allows each client to continually update its knowledge from task streams, enhancing the applicability of federated learning in real-world scenarios. However, FCL needs to address not only spatial data heterogeneity between clients but also temporal data heterogeneity between tasks. In this paper, empirical experiments demonstrate that such input-level heterogeneity significantly affects the model's internal parameters and outputs, leading to severe spatial-temporal catastrophic forgetting of local and previous knowledge. To this end, we propose Federated Tail Anchor (FedTA) to mix trainable Tail Anchor with the frozen output features to adjust their position in the feature space, thereby overcoming parameter-forgetting and output-forgetting. Three novel components are also included: Input Enhancement for improving the performance of pre-trained models on downstream tasks; Selective Input Knowledge Fusion for fusion of heterogeneous local knowledge on the server; and Best Global Prototype Selection for finding the best anchor point for each class in the feature space. Extensive experiments demonstrate that FedTA not only outperforms existing FCL methods but also effectively preserves the relative positions of features.