A Stochastic Block-coordinate Proximal Newton Method for Nonconvex Composite Minimization

TL;DR

This paper introduces a stochastic block-coordinate proximal Newton method for nonconvex composite minimization, featuring a line-search-free variant with proven convergence rates and demonstrated effectiveness through numerical experiments.

Contribution

It develops a novel stochastic block-coordinate proximal Newton algorithm with a line-search-free variant and establishes its convergence properties for nonconvex composite problems.

Findings

Convergence rates match those of inexact proximal Newton methods.

Global convergence rate of the residual mapping is established.

Numerical experiments demonstrate the algorithm's effectiveness.

Abstract

This paper presents a stochastic block-coordinate proximal Newton method for minimizing the sum of a blockwise Lipschitz-continuously differentiable function and a separable nonsmooth convex function. At each iteration, the method randomly selects one block and approximately solves a strongly convex regularized quadratic subproblem built from a second-order local model of the smooth part of the objective function, with a backtracking line search to ensure monotonicity of the objective. Under mild sampling assumptions, we show that its convergence properties match those of the inexact proximal Newton method. We further develop a line-search-free variant, where the strongly convex regularized quadratic subproblem is constructed using the Lipschitz constant of the gradient of the smooth component. For this variant, under a suitable parameter setting, we establish the global convergence rate of the residual mapping as well as the superlinear convergence rate of the iterates under the metric \(q\)-subregularity property with \(q > 1\) of the residual mapping for nonconvex composite problems. Under a suitable parameter setting, a more restrictive condition on the Hessian approximation, and the H\"olderian error bound condition (\(q\in(0, 1]\)) of the residual mapping, we also prove the local superlinear/quadratic convergence rate of both the residual mapping and the iterates for convex composite problems. Finally, numerical experiments are conducted to demonstrate the effectiveness and convergence behavior of the proposed algorithm.