MCMC Importance Sampling via Moreau-Yosida Envelopes

TL;DR

This paper introduces an importance sampling method using Moreau-Yosida envelopes to efficiently sample from non-differentiable priors in Bayesian models, enabling gradient-based MCMC and reducing estimator variance.

Contribution

It develops a novel importance sampling framework leveraging Moreau-Yosida envelopes, with theoretical analysis and practical algorithms for improved posterior sampling.

Findings

Asymptotic normality of the importance sampling estimator established.

Proposed method achieves lower variance estimators than existing proximal MCMC methods.

Effective in both low and high-dimensional settings.

Abstract

Non-differentiable priors are standard in modern parsimonious Bayesian models. Lack of differentiability, however, precludes gradient-based Markov chain Monte Carlo (MCMC) for posterior sampling. Recently proposed proximal MCMC approaches can partially remedy this limitation by using a differentiable approximation, constructed via Moreau-Yosida (MY) envelopes, to make proposals. In this work, we build an importance sampling paradigm by using the MY envelope as an importance distribution. Leveraging properties of the envelope, we establish asymptotic normality of the importance sampling estimator with an explicit expression for the asymptotic covariance matrix. Since the MY envelope density is smooth, it is amenable to gradient-based samplers. We provide sufficient conditions for geometric ergodicity of Metropolis-adjusted Langevin and Hamiltonian Monte Carlo algorithms, sampling from this importance distribution. Our numerical studies show that the proposed scheme can yield lower variance estimators compared to existing proximal MCMC alternatives, and is effective in low and high dimensions.