Bayesian Variational Federated Learning and Unlearning in Decentralized Networks

TL;DR

This paper introduces a Bayesian federated learning framework that enables decentralized collaborative training with quantifiable uncertainty and provides efficient unlearning protocols to remove individual contributions upon request.

Contribution

It develops federated variational inference methods for decentralized networks, integrating local free energy minimization and gossip communication for effective unlearning.

Findings

Efficient unlearning mechanisms in decentralized Bayesian federated learning.

Development of federated variational inference protocols for exponential-family models.

Demonstration of the proposed methods' effectiveness in unlearning tasks.

Abstract

Federated Bayesian learning offers a principled framework for the definition of collaborative training algorithms that are able to quantify epistemic uncertainty and to produce trustworthy decisions. Upon the completion of collaborative training, an agent may decide to exercise her legal "right to be forgotten", which calls for her contribution to the jointly trained model to be deleted and discarded. This paper studies federated learning and unlearning in a decentralized network within a Bayesian framework. It specifically develops federated variational inference (VI) solutions based on the decentralized solution of local free energy minimization problems within exponential-family models and on local gossip-driven communication. The proposed protocols are demonstrated to yield efficient unlearning mechanisms.