Bayesian Constraint Relaxation

TL;DR

This paper introduces a Bayesian constraint relaxation method that replaces sharp parameter constraints with a smooth exponential kernel, enabling broader modeling and easier computation with standard sampling algorithms.

Contribution

It proposes a novel continuous relaxation of constrained priors using an exponential kernel, allowing flexible modeling and compatibility with existing sampling techniques.

Findings

The relaxed distribution approximates the constrained distribution with quantifiable accuracy.

The method enables the use of standard samplers like Hamiltonian Monte Carlo.

Applications demonstrate broad modeling flexibility and computational advantages.

Abstract

Prior information often takes the form of parameter constraints. Bayesian methods include such information through prior distributions having constrained support. By using posterior sampling algorithms, one can quantify uncertainty without relying on asymptotic approximations. However, sharply constrained priors are (a) not necessary in some settings; and (b) tend to limit modeling scope to a narrow set of distributions that are tractable computationally. Inspired by the vast literature that replaces the slab-and-spike prior with a continuous approximation, we propose to replace the sharp indicator function of the constraint with an exponential kernel, thereby creating a close-to-constrained neighborhood within the Euclidean space in which the constrained subspace is embedded. This kernel decays with distance from the constrained space at a rate depending on a relaxation hyperparameter. By avoiding the sharp constraint, we enable use of off-the-shelf posterior sampling algorithms, such as Hamiltonian Monte Carlo, facilitating automatic computation in broad models. We study the constrained and relaxed distributions under multiple settings, and theoretically quantify their differences. We illustrate the method through multiple novel modeling examples.