Path-following methods for Maximum a Posteriori estimators in Bayesian hierarchical models: How estimates depend on hyperparameters

TL;DR

This paper introduces a path-following method for MAP estimators in Bayesian hierarchical models, enabling exploration of how solutions depend on hyperparameters and improving robustness in sparse inverse problems.

Contribution

It presents a predictor-corrector path-following algorithm for MAP estimators that explores the effect of hyperparameters in hierarchical models promoting sparsity.

Findings

Path following reveals solution sensitivity to hyperparameters.

Tracing paths from convex to non-convex regions finds stable sparse solutions.

Solutions obtained via path following are less error-prone than direct non-convex optimization.

Abstract

Maximum a posteriori (MAP) estimation, like all Bayesian methods, depends on prior assumptions. These assumptions are often chosen to promote specific features in the recovered estimate. The form of the chosen prior determines the shape of the posterior distribution, thus the behavior of the estimator and complexity of the associated optimization problem. Here, we consider a family of Gaussian hierarchical models with generalized gamma hyperpriors designed to promote sparsity in linear inverse problems. By varying the hyperparameters, we move continuously between priors that act as smoothed $\ell_p$ penalties with flexible $p$, smoothing, and scale. We then introduce a predictor-corrector method that tracks MAP solution paths as the hyperparameters vary. Path following allows a user to explore the space of possible MAP solutions and to test the sensitivity of solutions to changes in the prior assumptions. By tracing paths from a convex region to a non-convex region, the user can find local minimizers in strongly sparsity promoting regimes that are consistent with a convex relaxation derived using related prior assumptions. We show experimentally that these solutions. are less error prone than direct optimization of the non-convex problem.