Variational Bayes for Federated Continual Learning

TL;DR

This paper introduces FedBNN, a federated Bayesian neural network framework that effectively addresses catastrophic forgetting in federated continual learning by integrating knowledge from current and past data distributions.

Contribution

The paper presents FedBNN, a novel variational Bayesian approach for federated continual learning that maintains performance on historical data while learning from new data.

Findings

FedBNN outperforms existing methods in mitigating forgetting.

Experimental results show state-of-the-art performance across multiple datasets.

The framework effectively balances learning new information and retaining old knowledge.

Abstract

Federated continual learning (FCL) has received increasing attention due to its potential in handling real-world streaming data, characterized by evolving data distributions and varying client classes over time. The constraints of storage limitations and privacy concerns confine local models to exclusively access the present data within each learning cycle. Consequently, this restriction induces performance degradation in model training on previous data, termed "catastrophic forgetting". However, existing FCL approaches need to identify or know changes in data distribution, which is difficult in the real world. To release these limitations, this paper directs attention to a broader continuous framework. Within this framework, we introduce Federated Bayesian Neural Network (FedBNN), a versatile and efficacious framework employing a variational Bayesian neural network across all clients. Our method continually integrates knowledge from local and historical data distributions into a single model, adeptly learning from new data distributions while retaining performance on historical distributions. We rigorously evaluate FedBNN's performance against prevalent methods in federated learning and continual learning using various metrics. Experimental analyses across diverse datasets demonstrate that FedBNN achieves state-of-the-art results in mitigating forgetting.