Exemplar-condensed Federated Class-incremental Learning

TL;DR

This paper introduces ECoral, a federated learning method that uses exemplar condensation and inter-client sharing to effectively mitigate catastrophic forgetting in class-incremental learning scenarios with streaming data.

Contribution

The paper presents a novel exemplar condensation approach that maintains training gradient consistency and reduces heterogeneity, improving federated continual learning performance.

Findings

ECoral outperforms state-of-the-art methods in experiments.

It effectively mitigates catastrophic forgetting in federated class-incremental learning.

The method seamlessly integrates with existing approaches.

Abstract

We propose Exemplar-Condensed federated class-incremental learning (ECoral) to distil the training characteristics of real images from streaming data into informative rehearsal exemplars. The proposed method eliminates the limitations of exemplar selection in replay-based approaches for mitigating catastrophic forgetting in federated continual learning (FCL). The limitations particularly related to the heterogeneity of information density of each summarized data. Our approach maintains the consistency of training gradients and the relationship to past tasks for the summarized exemplars to represent the streaming data compared to the original images effectively. Additionally, our approach reduces the information-level heterogeneity of the summarized data by inter-client sharing of the disentanglement generative model. Extensive experiments show that our ECoral outperforms several state-of-the-art methods and can be seamlessly integrated with many existing approaches to enhance performance.