Federated Martingale Posterior Samping

TL;DR

This paper introduces Federated Martingale Posterior Sampling, a method enabling federated Bayesian neural networks to approximate predictive distributions without sharing raw data, improving calibration and matching centralized results.

Contribution

It proposes a one-shot federated sampling protocol using data embeddings, avoiding data sharing and maintaining accuracy and calibration.

Findings

FMP closely matches centralized predictive sampling results.

FMP significantly improves calibration over consensus baselines.

Experiments on MNIST, CIFAR-10, and CIFAR-100 validate effectiveness.

Abstract

Federated Bayesian neural networks require fixing a prior on the model parameters together with a likelihood. Eliciting meaningful priors on the weight space of modern overparameterized models is notoriously difficult, and misspecification of either component can severely degrade accuracy and calibration. Motivated by the rapid progress of predictive models such as large language models, the martingale posterior, also known as predictive Bayes, replaces the prior--likelihood pair with a predictive distribution and recovers parameter uncertainty by repeatedly drawing predictive samples and refitting the model. A direct federated implementation, however, would require clients to share the local data sets. This letter proposes {federated martingale posterior} (FMP) sampling, a one-shot embarrassingly parallel protocol in which each client uploads a small set of trainable data embeddings and the server runs the predictive sampler centrally. Experiments on MNIST, CIFAR-10, and CIFAR-100 show that FMP closely matches the centralized counterpart and significantly improves calibration over consensus-style baselines.