Optimal $k$-thresholding algorithms for sparse optimization problems

TL;DR

This paper introduces optimal $k$-thresholding algorithms that improve the stability and efficiency of sparse optimization solutions by directly linking thresholding to residual reduction, outperforming traditional methods.

Contribution

The paper proposes a novel optimal $k$-thresholding technique that enhances convergence and stability in sparse optimization, surpassing traditional hard thresholding methods.

Findings

Optimal $k$-thresholding algorithms are globally convergent under RIP.

Proposed algorithms outperform traditional hard thresholding methods.

Algorithms can surpass classic $ ext{l}_1$-minimization in many cases.

Abstract

The simulations indicate that the existing hard thresholding technique independent of the residual function may cause a dramatic increase or numerical oscillation of the residual. This inherit drawback of the hard thresholding renders the traditional thresholding algorithms unstable and thus generally inefficient for solving practical sparse optimization problems. How to overcome this weakness and develop a truly efficient thresholding method is a fundamental question in this field. The aim of this paper is to address this question by proposing a new thresholding technique based on the notion of optimal $k$-thresholding. The central idea for this new development is to connect the $k$-thresholding directly to the residual reduction during the course of algorithms. This leads to a natural design principle for the efficient thresholding methods. Under the restricted isometry property (RIP), we prove that the optimal thresholding based algorithms are globally convergent to the solution of sparse optimization problems. The numerical experiments demonstrate that when solving sparse optimization problems, the traditional hard thresholding methods have been significantly transcended by the proposed algorithms which can even outperform the classic $\ell_1$-minimization method in many situations.