Learning Fast Approximations of Sparse Nonlinear Regression

TL;DR

This paper introduces NLISTA, a neural network-based approach for sparse nonlinear regression that achieves linear convergence and outperforms existing methods, bridging a gap in the application of iterative algorithm unfolding to nonlinear problems.

Contribution

The paper proposes NLISTA, a novel deep learning method that extends iterative algorithm unfolding to sparse nonlinear regression with proven convergence properties.

Findings

NLISTA attains linear convergence under certain conditions.

Experimental results show NLISTA outperforms state-of-the-art methods.

Theoretical analysis confirms convergence rate of the proposed method.

Abstract

The idea of unfolding iterative algorithms as deep neural networks has been widely applied in solving sparse coding problems, providing both solid theoretical analysis in convergence rate and superior empirical performance. However, for sparse nonlinear regression problems, a similar idea is rarely exploited due to the complexity of nonlinearity. In this work, we bridge this gap by introducing the Nonlinear Learned Iterative Shrinkage Thresholding Algorithm (NLISTA), which can attain a linear convergence under suitable conditions. Experiments on synthetic data corroborate our theoretical results and show our method outperforms state-of-the-art methods.