Forget-SVGD: Particle-Based Bayesian Federated Unlearning

TL;DR

Forget-SVGD introduces a particle-based Bayesian federated unlearning method that efficiently removes data influence from federated models using local SVGD updates, improving upon existing schemes.

Contribution

The paper develops Forget-SVGD, a novel non-parametric Bayesian unlearning approach for federated learning, combining SVGD with federated updates for efficient data removal.

Findings

Outperforms training-from-scratch baselines in unlearning efficiency.

Effectively unlearns data with minimal impact on model performance.

Validates superiority over existing parametric unlearning methods.

Abstract

Variational particle-based Bayesian learning methods have the advantage of not being limited by the bias affecting more conventional parametric techniques. This paper proposes to leverage the flexibility of non-parametric Bayesian approximate inference to develop a novel Bayesian federated unlearning method, referred to as Forget-Stein Variational Gradient Descent (Forget-SVGD). Forget-SVGD builds on SVGD - a particle-based approximate Bayesian inference scheme using gradient-based deterministic updates - and on its distributed (federated) extension known as Distributed SVGD (DSVGD). Upon the completion of federated learning, as one or more participating agents request for their data to be "forgotten", Forget-SVGD carries out local SVGD updates at the agents whose data need to be "unlearned", which are interleaved with communication rounds with a parameter server. The proposed method is validated via performance comparisons with non-parametric schemes that train from scratch by excluding data to be forgotten, as well as with existing parametric Bayesian unlearning methods.