Proximal MCMC for Bayesian Inference of Constrained and Regularized Estimation

TL;DR

Proximal MCMC (ProxMCMC) is a flexible Bayesian inference framework that uses proximal algorithms and advanced sampling methods to handle constrained and regularized estimation problems, with data-adaptive parameter estimation.

Contribution

This paper extends ProxMCMC to be fully Bayesian by enabling data-driven estimation of all parameters, including regularization strength, and employs Hamiltonian Monte Carlo for high-dimensional problems.

Findings

Effective in high-dimensional Bayesian inference tasks

Handles constrained and regularized estimation flexibly

Demonstrates advantages over traditional methods

Abstract

This paper advocates proximal Markov Chain Monte Carlo (ProxMCMC) as a flexible and general Bayesian inference framework for constrained or regularized estimation. Originally introduced in the Bayesian imaging literature, ProxMCMC employs the Moreau-Yosida envelope for a smooth approximation of the total-variation regularization term, fixes variance and regularization strength parameters as constants, and uses the Langevin algorithm for the posterior sampling. We extend ProxMCMC to be fully Bayesian by providing data-adaptive estimation of all parameters including the regularization strength parameter. More powerful sampling algorithms such as Hamiltonian Monte Carlo are employed to scale ProxMCMC to high-dimensional problems. Analogous to the proximal algorithms in optimization, ProxMCMC offers a versatile and modularized procedure for conducting statistical inference on constrained and regularized problems. The power of ProxMCMC is illustrated on various statistical estimation and machine learning tasks, the inference of which is traditionally considered difficult from both frequentist and Bayesian perspectives.