The Variable Projected Augmented Lagrangian Method

TL;DR

This paper introduces the variable projected augmented Lagrangian (vpal) method for efficiently solving sparse inverse problems, with automatic regularization parameter selection and demonstrated effectiveness in imaging applications.

Contribution

The paper presents a novel vpal method that combines variable projection with augmented Lagrangian techniques for sparse inverse problems, including automatic regularization parameter selection.

Findings

Demonstrates computational efficiency in imaging problems

Provides an automatic regularization parameter selection approach

Offers a new perspective on l1 regularized inverse problems

Abstract

Inference by means of mathematical modeling from a collection of observations remains a crucial tool for scientific discovery and is ubiquitous in application areas such as signal compression, imaging restoration, and supervised machine learning. The inference problems may be solved using variational formulations that provide theoretically proven methods and algorithms. With ever-increasing model complexities and growing data size, new specially designed methods are urgently needed to recover meaningful quantifies of interest. We consider the broad spectrum of linear inverse problems where the aim is to reconstruct quantities with a sparse representation on some vector space; often solved using the (generalized) least absolute shrinkage and selection operator (lasso). The associated optimization problems have received significant attention, in particular in the early 2000's, because of their connection to compressed sensing and the reconstruction of solutions with favorable sparsity properties using augmented Lagrangians, alternating directions and splitting methods. We provide a new perspective on the underlying l1 regularized inverse problem by exploring the generalized lasso problem through variable projection methods. We arrive at our proposed variable projected augmented Lagrangian (vpal) method. We analyze this method and provide an approach for automatic regularization parameter selection based on a degrees of freedom argument. Further, we provide numerical examples demonstrating the computational efficiency for various imaging problems.