Gaussian Approximation and Output Analysis for High-Dimensional MCMC

TL;DR

This paper develops Gaussian approximation techniques for high-dimensional MCMC, providing bounds and methods to analyze output, estimate variance, and determine simulation effort, thereby aiding practitioners in high-dimensional statistical computations.

Contribution

It introduces novel Gaussian approximation results for high-dimensional MCMC, analyzing error dependencies on dimension and applying these to output analysis and termination criteria.

Findings

Gaussian approximation errors depend on dimension and feature space.

Termination time scales polynomially with dimension for desired precision.

Guidelines for standard error estimation and simulation effort in high-dimensional MCMC.

Abstract

The widespread use of Markov Chain Monte Carlo (MCMC) methods for high-dimensional applications has motivated research into the scalability of these algorithms with respect to the dimension of the problem. Despite this, numerous problems concerning output analysis in high-dimensional settings have remained unaddressed. We present novel quantitative Gaussian approximation results for a broad range of MCMC algorithms. Notably, we analyse the dependency of the obtained approximation errors on the dimension of both the target distribution and the feature space. We demonstrate how these Gaussian approximations can be applied in output analysis. This includes determining the simulation effort required to guarantee Markov chain central limit theorems and consistent estimation of the variance and effective sample size in high-dimensional settings. We give quantitative convergence bounds for termination criteria and show that the termination time of a wide class of MCMC algorithms scales polynomially in dimension while ensuring a desired level of precision. Our results offer guidance to practitioners for obtaining appropriate standard errors and deciding the minimum simulation effort of MCMC algorithms in both multivariate and high-dimensional settings.