Analog Sparse Approximation with Applications to Compressed Sensing

TL;DR

This paper proposes analog VLSI-based continuous dynamical systems for fast sparse approximation, significantly outperforming digital algorithms in speed, and applicable to various sparse regularization problems in compressed sensing.

Contribution

It introduces a novel analog approach inspired by neuroscience for solving sparse approximation problems efficiently in real-time.

Findings

Achieves signal recovery in 10-20 microseconds.

Supports high data rates of 50-100 kHz.

Applicable to multiple sparse regularization formulations.

Abstract

Recent research has shown that performance in signal processing tasks can often be significantly improved by using signal models based on sparse representations, where a signal is approximated using a small number of elements from a fixed dictionary. Unfortunately, inference in this model involves solving non-smooth optimization problems that are computationally expensive. While significant efforts have focused on developing digital algorithms specifically for this problem, these algorithms are inappropriate for many applications because of the time and power requirements necessary to solve large optimization problems. Based on recent work in computational neuroscience, we explore the potential advantages of continuous time dynamical systems for solving sparse approximation problems if they were implemented in analog VLSI. Specifically, in the simulated task of recovering synthetic and MRI data acquired via compressive sensing techniques, we show that these systems can potentially perform recovery at time scales of 10-20{\mu}s, supporting datarates of 50-100 kHz (orders of magnitude faster that digital algorithms). Furthermore, we show analytically that a wide range of sparse approximation problems can be solved in the same basic architecture, including approximate $\ell^p$ norms, modified $\ell^1$ norms, re-weighted $\ell^1$ and $\ell^2$, the block $\ell^1$ norm and classic Tikhonov regularization.