Proximal Diagonal Newton Methods for Composite Optimization Problems

TL;DR

This paper introduces proximal diagonal Newton methods for composite optimization, demonstrating their theoretical convergence advantages and effectiveness, especially in nonconvex problems, through numerical experiments.

Contribution

It presents a new proximal Newton-type method with a diagonal metric and provides theoretical convergence analysis and empirical validation.

Findings

Convergence rate suggests superiority over proximal gradient methods in some cases.

Numerical experiments confirm effectiveness, especially for nonconvex problems.

Proposed methods outperform traditional approaches in specific scenarios.

Abstract

This paper proposes new proximal Newton-type methods with a diagonal metric for solving composite optimization problems whose objective function is the sum of a twice continuously differentiable function and a proper closed directionally differentiable function. Although proximal Newton-type methods using diagonal metrics have been shown to be superior to the proximal gradient method numerically, no theoretical results have been obtained to suggest this superiority. Even though our proposed method is based on a simple idea, its convergence rate suggests an advantage over the proximal gradient method in certain situations. Numerical experiments show that our proposed algorithms are effective, especially in the nonconvex case.