Federated Generalised Variational Inference: A Robust Probabilistic Federated Learning Framework

TL;DR

FedGVI is a novel probabilistic federated learning framework that enhances robustness to model misspecification and improves predictive accuracy with efficient updates and calibrated uncertainty quantification.

Contribution

It generalizes previous FL methods by enabling robust, conjugate updates, reducing client computation, and providing theoretical guarantees of convergence and robustness.

Findings

FedGVI achieves improved robustness on synthetic and real datasets.

It provides unbiased predictions under model misspecification.

FedGVI demonstrates better predictive performance than existing methods.

Abstract

We introduce FedGVI, a probabilistic Federated Learning (FL) framework that is robust to both prior and likelihood misspecification. FedGVI addresses limitations in both frequentist and Bayesian FL by providing unbiased predictions under model misspecification, with calibrated uncertainty quantification. Our approach generalises previous FL approaches, specifically Partitioned Variational Inference (Ashman et al., 2022), by allowing robust and conjugate updates, decreasing computational complexity at the clients. We offer theoretical analysis in terms of fixed-point convergence, optimality of the cavity distribution, and provable robustness to likelihood misspecification. Further, we empirically demonstrate the effectiveness of FedGVI in terms of improved robustness and predictive performance on multiple synthetic and real world classification data sets.