Algorithm Unfolding for Block-sparse and MMV Problems with Reduced Training Overhead

TL;DR

This paper introduces a reduced-parameter neural network architecture for the MMV problem using algorithm unfolding, enabling effective sparse recovery with minimal training data and providing theoretical guarantees.

Contribution

It proposes a Kronecker-structured, reduced-size network for MMV problems, extending analytic weight methods and offering convergence guarantees with minimal training overhead.

Findings

Achieves near data-free optimization for MMV problems.

Provides explicit and closed-form solutions for network weights.

Demonstrates effectiveness on convolutional observation models.

Abstract

In this paper we consider algorithm unfolding for the Multiple Measurement Vector (MMV) problem in the case where only few training samples are available. Algorithm unfolding has been shown to empirically speed-up in a data-driven way the convergence of various classical iterative algorithms but for supervised learning it is important to achieve this with minimal training data. For this we consider learned block iterative shrinkage thresholding algorithm (LBISTA) under different training strategies. To approach almost data-free optimization at minimal training overhead the number of trainable parameters for algorithm unfolding has to be substantially reduced. We therefore explicitly propose a reduced-size network architecture based on the Kronecker structure imposed by the MMV observation model and present the corresponding theory in this context. To ensure proper generalization, we then extend the analytic weight approach by Lui et al to LBISTA and the MMV setting. Rigorous theoretical guarantees and convergence results are stated for this case. We show that the network weights can be computed by solving an explicit equation at the reduced MMV dimensions which also admits a closed-form solution. Towards more practical problems, we then consider convolutional observation models and show that the proposed architecture and the analytical weight computation can be further simplified and thus open new directions for convolutional neural networks. Finally, we evaluate the unfolded algorithms in numerical experiments and discuss connections to other sparse recovering algorithms.