Discovering Sparse Recovery Algorithms Using Neural Architecture Search

TL;DR

This paper introduces a neural architecture search framework to automatically rediscover and optimize sparse recovery algorithms like ISTA and FISTA, demonstrating adaptability across different data distributions and algorithm variants.

Contribution

The paper presents a novel meta-learning approach using NAS to automatically rediscover key sparse recovery algorithms from a large search space.

Findings

Successfully rediscovered ISTA and FISTA algorithms

Framework adapts to various data distributions

Can discover new algorithm variants

Abstract

The design of novel algorithms for solving inverse problems in signal processing is an incredibly difficult, heuristic-driven, and time-consuming task. In this short paper, we the idea of automated algorithm discovery in the signal processing context through meta-learning tools such as Neural Architecture Search (NAS). Specifically, we examine the Iterative Shrinkage Thresholding Algorithm (ISTA) and its accelerated Fast ISTA (FISTA) variant as candidates for algorithm rediscovery. We develop a meta-learning framework which is capable of rediscovering (several key elements of) the two aforementioned algorithms when given a search space of over 50,000 variables. We then show how our framework can apply to various data distributions and algorithms besides ISTA/FISTA.