Bayesian Trend Filtering via Proximal Markov Chain Monte Carlo

TL;DR

This paper introduces a new class of nondifferentiable priors called epigraph priors and extends proximal MCMC methods to automatically determine regularization parameters, demonstrated through a Bayesian trend filtering application.

Contribution

It develops epigraph priors and a gradient-based MCMC approach that automates regularization in Bayesian nonparametric regression.

Findings

Automates regularization parameter selection in Bayesian trend filtering.

Achieves tuning-free Bayesian inference with credible intervals.

Provides a fully Bayesian framework for nonparametric regression.

Abstract

Proximal Markov Chain Monte Carlo is a novel construct that lies at the intersection of Bayesian computation and convex optimization, which helped popularize the use of nondifferentiable priors in Bayesian statistics. Existing formulations of proximal MCMC, however, require hyperparameters and regularization parameters to be prespecified. In this work, we extend the paradigm of proximal MCMC through introducing a novel new class of nondifferentiable priors called epigraph priors. As a proof of concept, we place trend filtering, which was originally a nonparametric regression problem, in a parametric setting to provide a posterior median fit along with credible intervals as measures of uncertainty. The key idea is to replace the nonsmooth term in the posterior density with its Moreau-Yosida envelope, which enables the application of the gradient-based MCMC sampler Hamiltonian Monte Carlo. The proposed method identifies the appropriate amount of smoothing in a data-driven way, thereby automating regularization parameter selection. Compared with conventional proximal MCMC methods, our method is mostly tuning free, achieving simultaneous calibration of the mean, scale and regularization parameters in a fully Bayesian framework. Supplementary materials for this article are available online.