Federated Class-Incremental Learning with New-Class Augmented Self-Distillation

TL;DR

This paper introduces FedCLASS, a federated learning method that addresses the challenge of catastrophic forgetting in class-incremental scenarios by using self-distillation with augmented class scores, improving knowledge transfer and model performance.

Contribution

FedCLASS is a novel federated class-incremental learning approach that leverages self-distillation with new-class augmented scores, providing a theoretically grounded and empirically effective solution.

Findings

FedCLASS reduces average forgetting rate significantly.

FedCLASS achieves higher global accuracy than baseline methods.

Theoretical analysis supports the reliability of FedCLASS's approach.

Abstract

Federated Learning (FL) enables collaborative model training among participants while guaranteeing the privacy of raw data. Mainstream FL methodologies overlook the dynamic nature of real-world data, particularly its tendency to grow in volume and diversify in classes over time. This oversight results in FL methods suffering from catastrophic forgetting, where the trained models inadvertently discard previously learned information upon assimilating new data. In response to this challenge, we propose a novel Federated Class-Incremental Learning (FCIL) method, named \underline{Fed}erated \underline{C}lass-Incremental \underline{L}earning with New-Class \underline{A}ugmented \underline{S}elf-Di\underline{S}tillation (FedCLASS). The core of FedCLASS is to enrich the class scores of historical models with new class scores predicted by current models and utilize the combined knowledge for self-distillation, enabling a more sufficient and precise knowledge transfer from historical models to current models. Theoretical analyses demonstrate that FedCLASS stands on reliable foundations, considering scores of old classes predicted by historical models as conditional probabilities in the absence of new classes, and the scores of new classes predicted by current models as the conditional probabilities of class scores derived from historical models. Empirical experiments demonstrate the superiority of FedCLASS over four baseline algorithms in reducing average forgetting rate and boosting global accuracy.