Convergence analysis of a regularized Newton method with generalized regularization terms for convex optimization problems

TL;DR

This paper introduces a unified regularized Newton method with generalized regularization terms for convex optimization, achieving global and local superlinear convergence, and encompassing classical, cubic, and elastic net regularizations.

Contribution

It proposes a general framework for regularized Newton methods that includes new and existing regularizations, with proven convergence properties.

Findings

Achieves global $ ext{O}(k^{-2})$ convergence rate.

Attains local superlinear convergence.

Unifies multiple regularization approaches within one framework.

Abstract

This paper presents a regularized Newton method (RNM) with generalized regularization terms for unconstrained convex optimization problems. The generalized regularization includes quadratic, cubic, and elastic net regularizations as special cases. Therefore, the proposed method serves as a general framework that includes not only the classical and cubic RNMs but also a novel RNM with elastic net regularization. We show that the proposed RNM has the global $\mathcal{O}(k^{-2})$ and local superlinear convergence, which are the same as those of the cubic RNM.