Exact Convergence Analysis for Metropolis-Hastings Independence Samplers in Wasserstein Distances

TL;DR

This paper establishes exact convergence rates of the Metropolis-Hastings independence sampler in Wasserstein distances, providing new bounds and applications in Bayesian models, especially as sample size and dimension grow.

Contribution

It introduces precise convergence analysis in Wasserstein distances for the Metropolis-Hastings independence sampler, including new bounds, exact expressions, and applications to Bayesian models.

Findings

Exact convergence rates in Wasserstein distances match total variation rates.

New bounds on Wasserstein distance for arbitrary point initialization.

Application of the method to Bayesian quantile regression and high-dimensional binary response models.

Abstract

Under mild assumptions, we show the exact convergence rate in total variation is also exact in weaker Wasserstein distances for the Metropolis-Hastings independence sampler. We develop a new upper and lower bound on the worst-case Wasserstein distance when initialized from points. For an arbitrary point initialization, we show the convergence rate is the same and matches the convergence rate in total variation. We derive exact convergence expressions for more general Wasserstein distances when initialization is at a specific point. Using optimization, we construct a novel centered independent proposal to develop exact convergence rates in Bayesian quantile regression and many generalized linear model settings. We show the exact convergence rate can be upper bounded in Bayesian binary response regression (e.g. logistic and probit) when the sample size and dimension grow together.