An Inexact Proximal Newton Method for Nonconvex Composite Minimization

TL;DR

This paper introduces an inexact proximal Newton method for nonconvex composite optimization problems, demonstrating global convergence, local superlinear convergence, and competitive numerical performance on regression tasks.

Contribution

It develops a novel inexact proximal Newton algorithm with proven convergence rates for nonconvex problems, extending the applicability of Newton-type methods without line search.

Findings

Achieves an $ ext{O}(k^{-1/2})$ convergence rate.

Shows local superlinear convergence under certain regularity conditions.

Performs comparably to state-of-the-art methods in numerical experiments.

Abstract

In this paper, we propose an inexact proximal Newton-type method for nonconvex composite problems. We establish the global convergence rate of the order $\mathcal{O}(k^{-1/2})$ in terms of the minimal norm of the KKT residual mapping and the local superlinear convergence rate in terms of the sequence generated by the proposed algorithm under the higher-order metric $q$-subregularity property. When the Lipschitz constant of the corresponding gradient is known, we show that the proposed algorithm is well-defined without line search. Extensive numerical experiments on the $\ell_1$-regularized Student's $t$-regression and the group penalized Student's $t$-regression show that the performance of the proposed method is comparable to the state-of-the-art proximal Newton-type methods.