Mixing times of data-augmentation Gibbs samplers for high-dimensional probit regression

TL;DR

This paper analyzes the convergence rates of data-augmentation Gibbs samplers in high-dimensional Bayesian probit regression, providing explicit bounds and insights into how design and priors affect mixing times.

Contribution

It offers the first explicit non-asymptotic bounds on mixing times for these samplers, depending on data and prior parameters, with practical guidance for faster convergence.

Findings

Bounds depend on design matrix and prior precision.

Mixing times can remain bounded as data dimensions grow.

Guidance on prior choices for faster mixing.

Abstract

We investigate the convergence properties of popular data-augmentation samplers for Bayesian probit regression. Leveraging recent results on Gibbs samplers for log-concave targets, we provide simple and explicit non-asymptotic bounds on the associated mixing times (in Kullback-Leibler divergence). The bounds depend explicitly on the design matrix and the prior precision, while they hold uniformly over the vector of responses. We specialize the results for different regimes of statistical interest, when both the number of data points $n$ and parameters $p$ are large: in particular we identify scenarios where the mixing times remain bounded as $n,p\to\infty$, and ones where they do not. The results are shown to be tight (in the worst case with respect to the responses) and provide guidance on choices of prior distributions that provably lead to fast mixing. An empirical analysis based on coupling techniques suggests that the bounds are effective in predicting practically observed behaviours.