Tuning-Free Structured Sparse Recovery of Multiple Measurement Vectors using Implicit Regularization

TL;DR

This paper introduces a tuning-free method for jointly recovering sparse signals in MMV problems by using implicit regularization and overparameterization, eliminating the need for prior knowledge or parameter tuning.

Contribution

It proposes a novel overparameterized gradient descent approach with theoretical guarantees for support recovery without tuning, outperforming traditional methods.

Findings

Achieves comparable performance to tuned methods

Outperforms baselines when priors are unavailable

Provides formal convergence guarantees

Abstract

Recovering jointly sparse signals in the multiple measurement vectors (MMV) setting is a fundamental problem in machine learning, but traditional methods often require careful parameter tuning or prior knowledge of the sparsity of the signal and/or noise variance. We propose a tuning-free framework that leverages implicit regularization (IR) from overparameterization to overcome this limitation. Our approach reparameterizes the estimation matrix into factors that decouple the shared row-support from individual vector entries and applies gradient descent to a standard least-squares objective. We prove that with a sufficiently small and balanced initialization, the optimization dynamics exhibit a "momentum-like" effect where the true support grows significantly faster. Leveraging a Lyapunov-based analysis of the gradient flow, we further establish formal guarantees that the solution trajectory converges towards an idealized row-sparse solution. Empirical results demonstrate that our tuning-free approach achieves performance comparable to optimally tuned established methods. Furthermore, our framework significantly outperforms these baselines in scenarios where accurate priors are unavailable to the baselines.