Posterior Projection for Inference in Constrained Spaces

TL;DR

This paper introduces a generalized, efficient framework for constrained Bayesian inference by projecting unconstrained posteriors into the constrained space, supported by theoretical and practical validation.

Contribution

It proposes a novel projection-based approach for constrained Bayesian inference that is flexible, computationally efficient, and theoretically grounded.

Findings

The projected posterior distribution is mathematically well-founded.

The method achieves posterior consistency and contraction.

Practical applications demonstrate effectiveness and flexibility.

Abstract

Estimation of parameters that obey specific constraints is crucial in statistics and machine learning; for example, when parameters are required to satisfy boundedness, monotonicity, or linear inequalities. Traditional approaches impose these constraints via constraint-specific transformations or sampling approaches, or by truncating the posterior distribution. Such methods often result in computational challenges, limited flexibility, and a lack of generality. We propose a generalized framework for constrained Bayesian inference by projecting the unconstrained posterior distribution into the space of the parameter constraints, providing a computationally efficient and easily implementable solution for a large class of problems. We rigorously establish the theoretical foundations of the projected posterior distribution, as well as providing asymptotic results for posterior consistency, posterior contraction, and optimal coverage properties. Our methodology is validated through both theoretical arguments and practical applications.