Learned Greedy Method (LGM): A Novel Neural Architecture for Sparse Coding and Beyond

TL;DR

This paper introduces Learned Greedy Method (LGM), a neural network architecture based on unfolding the Orthogonal Matching Pursuit algorithm, enhancing interpretability and flexibility for sparse coding tasks.

Contribution

It presents the first unfolded and learned version of OMP, enabling dynamic layers and adaptive stopping, advancing neural network interpretability in sparse coding.

Findings

LGM achieves competitive sparse coding performance.

The architecture adapts to varying input complexities.

LGM demonstrates improved interpretability over black-box models.

Abstract

The fields of signal and image processing have been deeply influenced by the introduction of deep neural networks. These are successfully deployed in a wide range of real-world applications, obtaining state of the art results and surpassing well-known and well-established classical methods. Despite their impressive success, the architectures used in many of these neural networks come with no clear justification. As such, these are usually treated as "black box" machines that lack any kind of interpretability. A constructive remedy to this drawback is a systematic design of such networks by unfolding well-understood iterative algorithms. A popular representative of this approach is the Iterative Shrinkage-Thresholding Algorithm (ISTA) and its learned version -- LISTA, aiming for the sparse representations of the processed signals. In this paper we revisit this sparse coding task and propose an unfolded version of a greedy pursuit algorithm for the same goal. More specifically, we concentrate on the well-known Orthogonal-Matching-Pursuit (OMP) algorithm, and introduce its unfolded and learned version. Key features of our Learned Greedy Method (LGM) are the ability to accommodate a dynamic number of unfolded layers, and a stopping mechanism based on representation error, both adapted to the input. We develop several variants of the proposed LGM architecture and test some of them in various experiments, demonstrating their flexibility and efficiency.