Gradient Projection Newton Pursuit for Sparsity Constrained Optimization

TL;DR

This paper introduces a gradient projection Newton pursuit algorithm for sparsity-constrained optimization, combining hard-thresholding and Newton methods to achieve fast, globally convergent solutions under weak assumptions, especially effective in compressive sensing.

Contribution

It develops a novel algorithm that integrates hard-thresholding with Newton pursuit, offering improved convergence and performance in sparse optimization tasks.

Findings

Achieves global and quadratic convergence under standard assumptions.

Performs better than leading solvers in numerical experiments.

Requires weaker assumptions in compressive sensing applications.

Abstract

Hard-thresholding-based algorithms have seen various advantages for sparse optimization in controlling the sparsity and allowing for fast computation. Recent research shows that when techniques of the Newton-type methods are integrated, their numerical performance can be improved surprisingly. This paper develops a gradient projection Newton pursuit algorithm that mainly adopts the hard-thresholding operator and employs the Newton pursuit only when certain conditions are satisfied. The proposed algorithm is capable of converging globally and quadratically under the standard assumptions. When it comes to compressive sensing problems, the imposed assumptions are much weaker than those for many state-of-the-art algorithms. Moreover, extensive numerical experiments have demonstrated its high performance in comparison with the other leading solvers.