Convergence Rates for a Class of Estimators Based on Stein's Method

TL;DR

This paper establishes theoretical bounds on the convergence rates of Stein's method-based estimators, considering smoothness, dimension, and dependence, explaining their efficiency in low dimensions and limitations in high dimensions.

Contribution

It provides the first theoretical analysis of convergence rates for Stein's method estimators, including dependence and high-dimensional effects.

Findings

Gradient-based estimators converge rapidly in low dimensions.

High-dimensional settings exhibit a curse of dimensionality.

Theoretical bounds explain observed empirical behaviors.

Abstract

Gradient information on the sampling distribution can be used to reduce the variance of Monte Carlo estimators via Stein's method. An important application is that of estimating an expectation of a test function along the sample path of a Markov chain, where gradient information enables convergence rate improvement at the cost of a linear system which must be solved. The contribution of this paper is to establish theoretical bounds on convergence rates for a class of estimators based on Stein's method. Our analysis accounts for (i) the degree of smoothness of the sampling distribution and test function, (ii) the dimension of the state space, and (iii) the case of non-independent samples arising from a Markov chain. These results provide insight into the rapid convergence of gradient-based estimators observed for low-dimensional problems, as well as clarifying a curse-of-dimension that appears inherent to such methods.