Knowledge-Aware Evolution for Streaming Federated Continual Learning with Category Overlap and without Task Identifiers

TL;DR

This paper introduces FedKACE, a novel streaming federated continual learning method that effectively handles category overlap and absence of task identifiers, maintaining knowledge across non-stationary data streams.

Contribution

It proposes a new setting for streaming federated learning and introduces FedKACE, which includes adaptive model switching, gradient-balanced replay, and kernel spectral buffer for improved knowledge retention.

Findings

FedKACE outperforms existing methods in multiple streaming scenarios.

The adaptive inference mechanism improves personalization and generalization balance.

Kernel spectral buffer enhances knowledge retention across rounds.

Abstract

Federated Continual Learning (FCL) leverages inter-client collaboration to balance new knowledge acquisition and prior knowledge retention in non-stationary data. However, existing batch-based FCL methods lack adaptability to streaming scenarios featuring category overlap between old and new data and absent task identifiers, leading to indistinguishability of old and new knowledge, uncertain task assignments for samples, and knowledge confusion.To address this, we propose streaming federated continual learning setting: per federated learning (FL) round, clients process streaming data with disjoint samples and potentially overlapping categories without task identifiers, necessitating sustained inference capability for all prior categories after each FL round.Next, we introduce FedKACE: 1) an adaptive inference model switching mechanism that enables unidirectional switching from local model to global model to achieve a trade-off between personalization and generalization; 2) a adaptive gradient-balanced replay scheme that reconciles new knowledge learning and old knowledge retention under overlapping-class scenarios; 3) a kernel spectral boundary buffer maintenance that preserves high-information and high-boundary-influence samples to optimize cross-round knowledge retention. Experiments across multiple scenarios and regret analysis demonstrate the effectiveness of FedKACE.