Second-Order Methods with Cubic Regularization Under Inexact Information

TL;DR

This paper extends accelerated Newton's methods with cubic regularization to inexact second-order information, providing convergence guarantees and demonstrating practical speedups through numerical experiments.

Contribution

It introduces a generalized framework for second-order methods with inexact Hessians, establishing convergence rates and practical efficiency improvements.

Findings

Convergence rates depend explicitly on problem parameters.

Inexact Hessians can significantly accelerate algorithms in practice.

Theoretical complexity bounds are comparable to optimal first-order methods.

Abstract

In this paper, we generalize (accelerated) Newton's method with cubic regularization under inexact second-order information for (strongly) convex optimization problems. Under mild assumptions, we provide global rate of convergence of these methods and show the explicit dependence of the rate of convergence on the problem parameters. While the complexity bounds of our presented algorithms are theoretically worse than those of their exact counterparts, they are at least as good as those of the optimal first-order methods. Our numerical experiments also show that using inexact Hessians can significantly speed up the algorithms in practice.