Bayesian Inference with Projected Densities

TL;DR

This paper introduces a method for Bayesian inference that projects posterior distributions onto constraint sets, enabling efficient sampling and boundary analysis, especially useful in inverse problems and hierarchical models.

Contribution

It develops a novel projection-based approach for Bayesian posteriors on constrained parameter spaces, including efficient algorithms for Gaussian cases and hierarchical models.

Findings

Effective in Bayesian linear inverse problems

Enables boundary-focused posterior analysis

Demonstrated on deblurring and tomography tasks

Abstract

Constraints are a natural choice for prior information in Bayesian inference. In various applications, the parameters of interest lie on the boundary of the constraint set. In this paper, we use a method that implicitly defines a constrained prior such that the posterior assigns positive probability to the boundary of the constraint set. We show that by projecting posterior mass onto the constraint set, we obtain a new posterior with a rich probabilistic structure on the boundary of that set. If the original posterior is a Gaussian, then such a projection can be done efficiently. We apply the method to Bayesian linear inverse problems, in which case samples can be obtained by repeatedly solving constrained least squares problems, similar to a MAP estimate, but with perturbations in the data. When combined into a Bayesian hierarchical model and the constraint set is a polyhedral cone, we can derive a Gibbs sampler to efficiently sample from the hierarchical model. To show the effect of projecting the posterior, we applied the method to deblurring and computed tomography examples.