Modified Block Newton Algorithm for $\ell_0$-Regularized Optimization

TL;DR

This paper introduces a globally convergent Newton-type method with a line search for $\,\ell_0$-regularized sparse optimization, utilizing block-diagonal Jacobian computations to improve efficiency while maintaining convergence and quadratic rate.

Contribution

It presents a novel Newton-based algorithm with block-diagonal Jacobian approximation and regularization for $\,\ell_0$-regularized problems, ensuring global convergence and quadratic local convergence.

Findings

The method achieves global convergence.

Numerical results show high efficiency.

The algorithm maintains quadratic convergence rate.

Abstract

In this paper, we propose a globally convergent Newton type method to solve $\ell_0$ regularized sparse optimization problem. In fact, a line search strategy is applied to the Newton method to obtain global convergence. The Jacobian matrix of the original problem is a block upper triangular matrix. To reduce the computational burden, our method only requires the calculation of the block diagonal. We also introduced regularization to overcome matrix singularity. Although we only use the block-diagonal part of the Jacobian matrix, our algorithm still maintains global convergence and achieves a local quadratic convergence rate. Numerical results demonstrate the efficiency of our method.