Scalable Bayes via Barycenter in Wasserstein Space

TL;DR

This paper introduces a scalable Bayesian inference method that uses Wasserstein barycenters to combine subset posteriors, providing theoretical guarantees and demonstrating improved accuracy over existing methods in large-scale data scenarios.

Contribution

The paper proposes a novel divide-and-conquer Bayesian inference approach using Wasserstein barycenters, with theoretical guarantees and superior empirical performance.

Findings

The Wasserstein barycenter method approximates full posterior more accurately than competitors.

The approach scales Bayesian inference to massive datasets efficiently.

Empirical results on a movie ratings database validate the method's effectiveness.

Abstract

Divide-and-conquer based methods for Bayesian inference provide a general approach for tractable posterior inference when the sample size is large. These methods divide the data into smaller subsets, sample from the posterior distribution of parameters in parallel on all the subsets, and combine posterior samples from all the subsets to approximate the full data posterior distribution. The smaller size of any subset compared to the full data implies that posterior sampling on any subset is computationally more efficient than sampling from the true posterior distribution. Since the combination step takes negligible time relative to sampling, posterior computations can be scaled to massive data by dividing the full data into a sufficiently large number of data subsets. One such approach relies on the geometry of posterior distributions estimated across different subsets and combines them through their barycenter in a Wasserstein space of probability measures. We provide theoretical guarantees on the accuracy of approximation that are valid in many applications. We show that the geometric method approximates the full data posterior distribution better than its competitors across diverse simulations and reproduces known results when applied to a movie ratings database.