A Majorization-Minimization with Monte Carlo Approach for Hyperparameter Estimation

TL;DR

This paper introduces M3C, a novel algorithm combining majorization-minimization and Monte Carlo methods for efficient hyperparameter estimation in inverse problems with Gaussian priors.

Contribution

It develops a new M3C approach that simplifies hyperparameter optimization by replacing complex calculations with majorization and Monte Carlo estimations, ensuring convergence.

Findings

M3C converges to critical points with high probability.

Numerical examples demonstrate effectiveness in seismic tomography, super-resolution, and contaminant source problems.

Abstract

We consider inverse problems with linear forward models and Gaussian priors, but with unknown hyperparameters that may arise from the model, the noise, or the specification of the prior. We model this using a hierarchical Bayes framework resulting in a posterior distribution that is non-Gaussian, in general, and challenging to sample from. Consequently, we use an empirical Bayes framework for estimating the maximum a posteriori estimate of the hyperpameters by considering the marginalized posterior distribution. However, the optimization problem is also computationally challenging due to the need for repeated evaluation of log determinants. To address this issue, we propose a Majorization-Minimization with Monte Carlo approach, which we call M$^{3}$C, for hyperparameter estimation. Specifically, we replace the challenging optimization problem with a sequence of simpler ones by utilizing a majorization function (or majorant) for the log-determinant term, combined with a Monte Carlo estimator to approximate the majorant. We provide theoretical results, showing that under certain assumptions, the M$^{3}$C iterates converge with high probability to a critical point of the original cost function. A variety of numerical examples are provided from seismic tomography, super-resolution imaging, and contaminant source identification.