Variational Garrote for Sparse Inverse Problems

TL;DR

This paper compares traditional L1 regularization with the Variational Garrote, a probabilistic method approximating L0 sparsity, across various inverse problem tasks, showing VG often yields better generalization and stability especially in highly underdetermined settings.

Contribution

It introduces a unified experimental framework to compare sparsity priors, demonstrating the advantages of Variational Garrote over L1 regularization in sparse inverse problems.

Findings

VG achieves lower generalization error

VG shows improved stability in underdetermined regimes

Sparsity priors closer to spike-and-slab are advantageous

Abstract

Sparse regularization plays a central role in solving inverse problems arising from incomplete or corrupted measurements. Different regularizers correspond to different prior assumptions about the structure of the unknown signal, and reconstruction performance depends on how well these priors match the intrinsic sparsity of the data. This work investigates the effect of sparsity priors in inverse problems by comparing conventional L1 regularization with the Variational Garrote (VG), a probabilistic method that approximates L0 sparsity through variational binary gating variables. A unified experimental framework is constructed across multiple reconstruction tasks including signal resampling, signal denoising, and sparse-view computed tomography. To enable consistent comparison across models with different parameterizations, regularization strength is swept across wide ranges and reconstruction behavior is analyzed through train-generalization error curves. Experiments reveal characteristic bias-variance tradeoff patterns across tasks and demonstrate that VG frequently achieves lower minimum generalization error and improved stability in strongly underdetermined regimes where accurate support recovery is critical. These results suggest that sparsity priors closer to spike-and-slab structure can provide advantages when the underlying coefficient distribution is strongly sparse. The study highlights the importance of prior-data alignment in sparse inverse problems and provides empirical insights into the behavior of variational L0-type methods across different information bottlenecks.