Convergence analysis of data augmentation algorithms in Bayesian lasso models with log-concave likelihoods

TL;DR

This paper analyzes the convergence rates of data augmentation algorithms for Bayesian lasso models with log-concave likelihoods, providing bounds on mixing times for various models.

Contribution

It introduces a generic convergence bound for these algorithms using isoperimetric inequalities and applies it to multiple Bayesian lasso models, deriving explicit mixing time bounds.

Findings

Proven mixing times are polynomial in sample size and dimension.

Derived explicit bounds for Bayesian probit, logistic, and heteroskedastic Gaussian models.

Provided conditions under which the algorithms converge efficiently.

Abstract

We study the convergence properties of a class of data augmentation algorithms targeting posterior distributions of Bayesian lasso models with log-concave likelihoods. Leveraging isoperimetric inequalities, we derive a generic convergence bound for this class of algorithms and apply it to Bayesian probit, logistic, and heteroskedastic Gaussian linear lasso models. Under feasible initializations, the mixing times for the probit and logistic models are of order $O[(p+n)^3 (pn^{1-c} + n)]$, up to logarithmic factors, where $n$ is the sample size, $p$ is the dimension of the regression coefficients, and $c \in [0,1]$ is determined by the lasso penalty parameter. The mixing time for the heteroskedastic Gaussian model is $O[n(n+p)^3 (p n^{1-c} + n)]$, up to logarithmic factors.