Frequentist Guarantees of Distributed (Non)-Bayesian Inference

TL;DR

This paper analyzes the frequentist properties of distributed Bayesian inference, demonstrating that it can achieve parametric efficiency and robustness under certain network conditions, with implications for various models.

Contribution

It provides the first comprehensive frequentist guarantees for distributed Bayesian inference, including posterior consistency, normality, and contraction rates, extending to time-varying graphs and specific models.

Findings

Distributed Bayesian inference retains parametric efficiency.

Robustness in uncertainty quantification is improved.

Communication graph design affects posterior contraction rates.

Abstract

Motivated by the need to analyze large, decentralized datasets, distributed Bayesian inference has become a critical research area across multiple fields, including statistics, electrical engineering, and economics. This paper establishes Frequentist properties, such as posterior consistency, asymptotic normality, and posterior contraction rates, for the distributed (non-)Bayes Inference problem among agents connected via a communication network. Our results show that, under appropriate assumptions on the communication graph, distributed Bayesian inference retains parametric efficiency while enhancing robustness in uncertainty quantification. We also explore the trade-off between statistical efficiency and communication efficiency by examining how the design and size of the communication graph impact the posterior contraction rate. Furthermore, We extend our analysis to time-varying graphs and apply our results to exponential family models, distributed logistic regression, and decentralized detection models.