Quartic Regularity

TL;DR

This paper introduces new second-order optimization methods with linear convergence for convex quartic polynomial problems, leveraging quartic regularity and high-order proximal schemes to achieve faster convergence rates.

Contribution

It develops a novel framework for convex quartic problems with a new regularity condition, enabling globally linearly convergent second-order methods.

Findings

Global linear convergence rate for the proposed methods.

Methods achieve convergence rates of  O(k^{-p}) with p=3,4,5.

Applicable to high-order proximal-point schemes.

Abstract

In this paper, we propose new linearly convergent second-order methods for minimizing convex quartic polynomials. This framework is applied for designing optimization schemes, which can solve general convex problems satisfying a new condition of quartic regularity. It assumes positive definiteness and boundedness of the fourth derivative of the objective function. For such problems, an appropriate quartic regularization of Damped Newton Method has global linear rate of convergence. We discuss several important consequences of this result. In particular, it can be used for constructing new second-order methods in the framework of high-order proximal-point schemes. These methods have convergence rate $\tilde O(k^{-p})$, where $k$ is the iteration counter, $p$ is equal to 3, 4, or 5, and tilde indicates the presence of logarithmic factors in the complexity bounds for the auxiliary problems, which are solved at each iteration of the schemes.