Rapid mixing of a Markov chain for an exponentially weighted aggregation estimator

TL;DR

This paper introduces a simplified and improved method for proving rapid mixing of Markov chains, demonstrated through an application to an exponentially weighted aggregation estimator in high-dimensional Bayesian models.

Contribution

It proposes a modified approach to the path method that simplifies analysis and enhances convergence rates for Markov chains in high-dimensional settings.

Findings

Simplified theoretical proof of rapid mixing

Improved convergence rate for the Markov chain

Application to exponentially weighted aggregation estimator

Abstract

The Metropolis-Hastings method is often used to construct a Markov chain with a given $\pi$ as its stationary distribution. The method works even if $\pi$ is known only up to an intractable constant of proportionality. Polynomial time convergence results for such chains (rapid mixing) are hard to obtain for high dimensional probability models where the size of the state space potentially grows exponentially with the model dimension. In a Bayesian context, Yang, Wainwright, and Jordan (2016) (=YWJ) used the path method to prove rapid mixing for high dimensional linear models. This paper proposes a modification of the YWJ approach that simplifies the theoretical argument and improves the rate of convergence. The new approach is illustrated by an application to an exponentially weighted aggregation estimator.