Variable Projected Augmented Lagrangian Methods for Generalized Lasso Problems

TL;DR

This paper introduces variable projected augmented Lagrangian (VPAL) methods that improve the speed and accuracy of solving generalized nonlinear Lasso problems, extending to nonlinear inverse problems with state-of-the-art results.

Contribution

The paper develops VPAL methods that eliminate nonsmooth variables via soft-thresholding, providing convergence guarantees and significant acceleration, especially for nonlinear inverse problems.

Findings

Outperforms traditional methods in phase retrieval and MRI.

Achieves state-of-the-art results in deblurring, inpainting, and tomography.

Provides convergence guarantees for both standard and preconditioned VPAL.

Abstract

We introduce variable projected augmented Lagrangian (VPAL) methods for solving generalized nonlinear Lasso problems with improved speed and accuracy. By eliminating the nonsmooth variable via soft-thresholding, VPAL transforms the problem into a smooth reduced formulation. For linear models, we develop a preconditioned variant that mimics Newton-type updates and yields significant acceleration. We prove convergence guarantees for both standard and preconditioned VPAL under mild assumptions and show that variable projection leads to sharper convergence and higher solution quality. The method seamlessly extends to nonlinear inverse problems, where it outperforms traditional approaches in applications such as phase retrieval and contrast enhanced MRI (LIP-CAR). Across tasks including deblurring, inpainting, and sparse-view tomography, VPAL consistently delivers state-of-the-art reconstructions, positioning variable projection as a powerful tool for modern large-scale inverse problems.