Estimating Nuisance Parameters in Inverse Problems

TL;DR

This paper generalizes the variable projection method to efficiently estimate nuisance parameters in large-scale inverse problems, improving primary parameter recovery across various applications.

Contribution

It extends variable projection techniques to a broad class of ML and MAP problems with nuisance parameters, enabling their integration into large-scale inverse problem frameworks.

Findings

Improved recovery of primary parameters in large-scale inverse problems.

Effective estimation of nuisance parameters like variance and degrees of freedom.

Compatible with existing algorithms with minimal modifications.

Abstract

Many inverse problems include nuisance parameters which, while not of direct interest, are required to recover primary parameters. Structure present in these problems allows efficient optimization strategies - a well known example is variable projection, where nonlinear least squares problems which are linear in some parameters can be very efficiently optimized. In this paper, we extend the idea of projecting out a subset over the variables to a broad class of maximum likelihood (ML) and maximum a posteriori likelihood (MAP) problems with nuisance parameters, such as variance or degrees of freedom. As a result, we are able to incorporate nuisance parameter estimation into large-scale constrained and unconstrained inverse problem formulations. We apply the approach to a variety of problems, including estimation of unknown variance parameters in the Gaussian model, degree of freedom (d.o.f.) parameter estimation in the context of robust inverse problems, automatic calibration, and optimal experimental design. Using numerical examples, we demonstrate improvement in recovery of primary parameters for several large- scale inverse problems. The proposed approach is compatible with a wide variety of algorithms and formulations, and its implementation requires only minor modifications to existing algorithms.