Federated Continual Learning Goes Online: Uncertainty-Aware Memory Management for Vision Tasks and Beyond

TL;DR

This paper introduces an online, uncertainty-aware memory management method for federated continual learning that effectively reduces catastrophic forgetting across multiple modalities without requiring offline data storage or multiple training epochs.

Contribution

It proposes a novel uncertainty-based memory retrieval technique using Bregman Information to mitigate forgetting in online federated learning across various data modalities.

Findings

Reduces catastrophic forgetting effectively in online federated learning.

Maintains data confidentiality and communication efficiency.

Applicable to multiple data modalities beyond vision.

Abstract

Given the ability to model more realistic and dynamic problems, Federated Continual Learning (FCL) has been increasingly investigated recently. A well-known problem encountered in this setting is the so-called catastrophic forgetting, for which the learning model is inclined to focus on more recent tasks while forgetting the previously learned knowledge. The majority of the current approaches in FCL propose generative-based solutions to solve said problem. However, this setting requires multiple training epochs over the data, implying an offline setting where datasets are stored locally and remain unchanged over time. Furthermore, the proposed solutions are tailored for vision tasks solely. To overcome these limitations, we propose a new approach to deal with different modalities in the online scenario where new data arrive in streams of mini-batches that can only be processed once. To solve catastrophic forgetting, we propose an uncertainty-aware memory-based approach. Specifically, we suggest using an estimator based on the Bregman Information (BI) to compute the model's variance at the sample level. Through measures of predictive uncertainty, we retrieve samples with specific characteristics, and - by retraining the model on such samples - we demonstrate the potential of this approach to reduce the forgetting effect in realistic settings while maintaining data confidentiality and competitive communication efficiency compared to state-of-the-art approaches.