MIST: L0 Sparse Linear Regression with Momentum

TL;DR

This paper introduces MIST, a new algorithm for sparse linear regression with an $l_0$ penalty, demonstrating improved performance and convergence properties on large datasets.

Contribution

The paper develops a Momentumized Iterative Shrinkage Thresholding (MIST) algorithm for non-convex $l_0$ regularized problems, with proven convergence to a local minimizer.

Findings

MIST outperforms existing methods in large-scale experiments.

The algorithm converges reliably to a local minimum.

Simulations confirm the effectiveness of the proposed approach.

Abstract

Significant attention has been given to minimizing a penalized least squares criterion for estimating sparse solutions to large linear systems of equations. The penalty is responsible for inducing sparsity and the natural choice is the so-called $l_0$ norm. In this paper we develop a Momentumized Iterative Shrinkage Thresholding (MIST) algorithm for minimizing the resulting non-convex criterion and prove its convergence to a local minimizer. Simulations on large data sets show superior performance of the proposed method to other methods.