Bayesian Personalized Federated Learning with Shared and Personalized Uncertainty Representations

TL;DR

This paper introduces a Bayesian federated learning framework that decomposes shared and personalized uncertainty representations to better handle client heterogeneity and improve model convergence in federated systems.

Contribution

It proposes a novel BPFL framework with a Bayesian neural network that jointly learns shared and personalized uncertainties, incorporating continual prior updates for enhanced performance.

Findings

BPFed effectively models shared and personalized uncertainties.

The framework accelerates convergence and mitigates catastrophic forgetting.

Experimental results outperform diversified baselines.

Abstract

Bayesian personalized federated learning (BPFL) addresses challenges in existing personalized FL (PFL). BPFL aims to quantify the uncertainty and heterogeneity within and across clients towards uncertainty representations by addressing the statistical heterogeneity of client data. In PFL, some recent preliminary work proposes to decompose hidden neural representations into shared and local components and demonstrates interesting results. However, most of them do not address client uncertainty and heterogeneity in FL systems, while appropriately decoupling neural representations is challenging and often ad hoc. In this paper, we make the first attempt to introduce a general BPFL framework to decompose and jointly learn shared and personalized uncertainty representations on statistically heterogeneous client data over time. A Bayesian federated neural network BPFed instantiates BPFL by jointly learning cross-client shared uncertainty and client-specific personalized uncertainty over statistically heterogeneous and randomly participating clients. We further involve continual updating of prior distribution in BPFed to speed up the convergence and avoid catastrophic forgetting. Theoretical analysis and guarantees are provided in addition to the experimental evaluation of BPFed against the diversified baselines.