A Divide and Conquer Algorithm of Bayesian Density Estimation

TL;DR

This paper introduces a divide and conquer Bayesian density estimation method that efficiently handles large datasets by splitting data, with theoretical guarantees and demonstrated advantages over existing methods in simulations and real data.

Contribution

It proposes a novel Bayesian divide and conquer approach for density estimation that maintains statistical efficiency and reduces computation time.

Findings

Posterior contraction rate remains nearly the same as the full data analysis.

Simulation studies show superior performance over the WASP estimator.

Application to GWAS data demonstrates practical advantages.

Abstract

Data sets for statistical analysis become extremely large even with some difficulty of being stored on one single machine. Even when the data can be stored in one machine, the computational cost would still be intimidating. We propose a divide and conquer solution to density estimation using Bayesian mixture modeling including the infinite mixture case. The methodology can be generalized to other application problems where a Bayesian mixture model is adopted. The proposed prior on each machine or subsample modifies the original prior on both mixing probabilities as well as on the rest of parameters in the distributions being mixed. The ultimate estimator is obtained by taking the average of the posterior samples corresponding to the proposed prior on each subset. Despite the tremendous reduction in time thanks to data splitting, the posterior contraction rate of the proposed estimator stays the same (up to a log factor) as that of the original prior when the data is analyzed as a whole. Simulation studies also justify the competency of the proposed method compared to the established WASP estimator in the finite dimension case. In addition, one of our simulations is performed in a shape constrained deconvolution context and reveals promising results. The application to a GWAS data set reveals the advantage over a naive method that uses the original prior.