Distance-to-Set Priors and Constrained Bayesian Inference

TL;DR

This paper introduces a Bayesian approach to constrained learning using distance-to-set priors, enabling uncertainty quantification and improved statistical properties in constrained inference tasks.

Contribution

It develops smooth distance-to-set priors for Bayesian constrained learning, providing a theoretically optimal and practically effective method for uncertainty quantification.

Findings

Posteriors are well-defined for uncertainty quantification.

Method is optimal in an information geometric sense for finite penalties.

Effective within gradient-based MCMC, demonstrated on simulated and real data.

Abstract

Constrained learning is prevalent in many statistical tasks. Recent work proposes distance-to-set penalties to derive estimators under general constraints that can be specified as sets, but focuses on obtaining point estimates that do not come with corresponding measures of uncertainty. To remedy this, we approach distance-to-set regularization from a Bayesian lens. We consider a class of smooth distance-to-set priors, showing that they yield well-defined posteriors toward quantifying uncertainty for constrained learning problems. We discuss relationships and advantages over prior work on Bayesian constraint relaxation. Moreover, we prove that our approach is optimal in an information geometric-sense for finite penalty parameters $\rho$, and enjoys favorable statistical properties when $\rho\to\infty$. The method is designed to perform effectively within gradient-based MCMC samplers, as illustrated on a suite of simulated and real data applications.