Parallel Markov Chain Monte Carlo for Non-Gaussian Posterior Distributions

TL;DR

This paper introduces a new density product method for combining subset posterior samples in parallel MCMC, effectively estimating non-Gaussian full data posteriors in big data Bayesian analysis.

Contribution

The paper presents a novel direct density product approach that outperforms existing methods for non-Gaussian posteriors in parallel MCMC settings.

Findings

Outperforms existing methods in non-Gaussian posterior approximation

Effective for high-dimensional models with fixed parameter dimension

Applicable to large-scale Bayesian data analysis

Abstract

Recent developments in big data and analytics research have produced an abundance of large data sets that are too big to be analyzed in their entirety, due to limits on computer memory or storage capacity. To address these issues, communication-free parallel Markov chain Monte Carlo (MCMC) methods have been developed for Bayesian analysis of big data. These methods partition data into manageable subsets, perform independent Bayesian MCMC analysis on each subset, and combine the subset posterior samples to estimate the full data posterior. Current approaches to combining subset posterior samples include sample averaging, weighted averaging, and kernel smoothing techniques. Although these methods work well for Gaussian posteriors, they are not well-suited to non-Gaussian posterior distributions. Here, we develop a new direct density product method for combining subset marginal posterior samples to estimate full data marginal posterior densities. Using a commonly-implemented distance metric, we show in simulation studies of Bayesian models with non-Gaussian posteriors that our method outperforms the existing methods in approximating the full data marginal posteriors. Since our method estimates only marginal densities, there is no limitation on the number of model parameters analyzed. Our procedure is suitable for Bayesian models with unknown parameters with fixed dimension in continuous parameter spaces.