Bayesian Fusion: Scalable unification of distributed statistical analyses

TL;DR

This paper introduces Bayesian Fusion, a scalable and exact method for unifying distributed statistical analyses, addressing limitations of approximate methods especially in large or non-concordant analysis scenarios.

Contribution

It extends the Fusion methodology using sequential Monte Carlo, enabling robust, exact, and scalable unification of multiple distributed analyses.

Findings

The method is robust to increasing number of analyses.

It maintains accuracy even when analyses do not concur.

Computationally competitive with approximate schemes.

Abstract

There has recently been considerable interest in addressing the problem of unifying distributed statistical analyses into a single coherent inference. This problem naturally arises in a number of situations, including in big-data settings, when working under privacy constraints, and in Bayesian model choice. The majority of existing approaches have relied upon convenient approximations of the distributed analyses. Although typically being computationally efficient, and readily scaling with respect to the number of analyses being unified, approximate approaches can have significant shortcomings -- the quality of the inference can degrade rapidly with the number of analyses being unified, and can be substantially biased even when unifying a small number of analyses that do not concur. In contrast, the recent Fusion approach of Dai et al. (2019) is a rejection sampling scheme which is readily parallelisable and is exact (avoiding any form of approximation other than Monte Carlo error), albeit limited in applicability to unifying a small number of low-dimensional analyses. In this paper we introduce a practical Bayesian Fusion approach. We extend the theory underpinning the Fusion methodology and, by embedding it within a sequential Monte Carlo algorithm, we are able to recover the correct target distribution. By means of extensive guidance on the implementation of the approach, we demonstrate theoretically and empirically that Bayesian Fusion is robust to increasing numbers of analyses, and coherently unifying analyses which do not concur. This is achieved while being computationally competitive with approximate schemes.