Subspace Newton's Method for $\ell_0$-Regularized Optimization Problems with Box Constraint

TL;DR

This paper proposes a novel Newton-based method for solving box-constrained $ ext{l}_0$-regularized sparse optimization problems, establishing convergence properties and demonstrating efficiency through experiments.

Contribution

It introduces the concept of $ au$-stationary points, combines Newton's method with proximal gradient updates, and proves convergence under mild conditions.

Findings

Global convergence is established.

Local quadratic convergence rate is proven.

Experimental results show high efficiency.

Abstract

This paper investigates the box-constrained $\ell_0$-regularized sparse optimization problem. We introduce the concept of a $\tau$-stationary point and establish its connection to the local and global minima of the box-constrained $\ell_0$-regularized sparse optimization problem. We utilize the $\tau$-stationary points to define the support set, which we divide into active and inactive components. Subsequently, the Newton's method is employed to update the non-active variables, while the proximal gradient method is utilized to update the active variables. If the Newton's method fails, we use the proximal gradient step to update all variables. Under some mild conditions, we prove the global convergence and the local quadratic convergence rate. Finally, experimental results demonstrate the efficiency of our method.