Compressed Particle-Based Federated Bayesian Learning and Unlearning

TL;DR

This paper introduces compressed particle-based Bayesian federated learning and unlearning protocols that maintain calibration benefits despite communication bandwidth constraints, using quantization and sparsification techniques.

Contribution

It proposes novel compressed Bayesian FL and unlearning methods that reduce communication overhead while preserving the advantages of Bayesian approaches.

Findings

Bayesian FL benefits are robust under bandwidth constraints

Quantization and sparsification effectively reduce communication costs

Experimental results validate the approach's effectiveness

Abstract

Conventional frequentist FL schemes are known to yield overconfident decisions. Bayesian FL addresses this issue by allowing agents to process and exchange uncertainty information encoded in distributions over the model parameters. However, this comes at the cost of a larger per-iteration communication overhead. This letter investigates whether Bayesian FL can still provide advantages in terms of calibration when constraining communication bandwidth. We present compressed particle-based Bayesian FL protocols for FL and federated "unlearning" that apply quantization and sparsification across multiple particles. The experimental results confirm that the benefits of Bayesian FL are robust to bandwidth constraints.