Probabilistic Federated Learning on Uncertain and Heterogeneous Data with Model Personalization

TL;DR

This paper introduces Meta-BayFL, a personalized probabilistic federated learning framework that models uncertainty with Bayesian neural networks and uses meta-learning to improve training on uncertain, heterogeneous data, achieving higher accuracy.

Contribution

It proposes a novel combination of Bayesian neural networks and meta-learning for personalized federated learning under data uncertainty and heterogeneity.

Findings

Meta-BayFL outperforms state-of-the-art FL methods by up to 7.42% in test accuracy.

The framework stabilizes training on small, noisy datasets with uncertainty modeling.

Extensive experiments validate the effectiveness and resource feasibility of Meta-BayFL.

Abstract

Conventional federated learning (FL) frameworks often suffer from training degradation due to data uncertainty and heterogeneity across local clients. Probabilistic approaches such as Bayesian neural networks (BNNs) can mitigate this issue by explicitly modeling uncertainty, but they introduce additional runtime, latency, and bandwidth overhead that has rarely been studied in federated settings. To address these challenges, we propose Meta-BayFL, a personalized probabilistic FL method that combines meta-learning with BNNs to improve training under uncertain and heterogeneous data. The framework is characterized by three main features: (1) BNN-based client models incorporate uncertainty across hidden layers to stabilize training on small and noisy datasets, (2) meta-learning with adaptive learning rates enables personalized updates that enhance local training under non-IID conditions, and (3) a unified probabilistic and personalized design improves the robustness of global model aggregation. We provide a theoretical convergence analysis and characterize the upper bound of the global model over communication rounds. In addition, we evaluate computational costs (runtime, latency, and communication) and discuss the feasibility of deployment on resource-constrained devices such as edge nodes and IoT systems. Extensive experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet show that Meta-BayFL consistently outperforms state-of-the-art methods, including both standard and personalized FL approaches (e.g., pFedMe, Ditto, FedFomo), with up to 7.42\% higher test accuracy.