No One Left Behind: Real-World Federated Class-Incremental Learning

TL;DR

This paper introduces a novel federated learning approach called LGA that effectively mitigates catastrophic forgetting of old categories in class-incremental learning scenarios, especially under non-IID data distributions.

Contribution

The paper proposes the Local-Global Anti-forgetting (LGA) model, combining category-balanced gradient compensation, semantic distillation, and a proxy server with prototype augmentation to address local and global forgetting in federated class-incremental learning.

Findings

LGA outperforms existing methods on benchmark datasets.

The proposed losses balance forgetting speeds of different categories.

Prototype augmentation improves robustness to non-IID data.

Abstract

Federated learning (FL) is a hot collaborative training framework via aggregating model parameters of decentralized local clients. However, most FL methods unreasonably assume data categories of FL framework are known and fixed in advance. Moreover, some new local clients that collect novel categories unseen by other clients may be introduced to FL training irregularly. These issues render global model to undergo catastrophic forgetting on old categories, when local clients receive new categories consecutively under limited memory of storing old categories. To tackle the above issues, we propose a novel Local-Global Anti-forgetting (LGA) model. It ensures no local clients are left behind as they learn new classes continually, by addressing local and global catastrophic forgetting. Specifically, considering tackling class imbalance of local client to surmount local forgetting, we develop a category-balanced gradient-adaptive compensation loss and a category gradient-induced semantic distillation loss. They can balance heterogeneous forgetting speeds of hard-to-forget and easy-to-forget old categories, while ensure consistent class-relations within different tasks. Moreover, a proxy server is designed to tackle global forgetting caused by Non-IID class imbalance between different clients. It augments perturbed prototype images of new categories collected from local clients via self-supervised prototype augmentation, thus improving robustness to choose the best old global model for local-side semantic distillation loss. Experiments on representative datasets verify superior performance of our model against comparison methods. The code is available at https://github.com/JiahuaDong/LGA.