Adaptive Cubic Regularization Methods with Dynamic Inexact Hessian Information and Applications to Finite-Sum Minimization

TL;DR

This paper introduces a new adaptive cubic regularization method that dynamically uses inexact Hessian information, providing theoretical guarantees and practical efficiency for large-scale nonconvex finite-sum minimization problems.

Contribution

It proposes a novel variant of the ARC method with dynamic inexact Hessian, offering improved theoretical analysis and applicability to large-scale problems with subsampled Hessians.

Findings

The method guarantees optimal worst-case evaluation bounds.

Effective for large-scale finite-sum minimization with subsampled Hessians.

Numerical experiments demonstrate practical performance on synthetic and real datasets.

Abstract

We consider the Adaptive Regularization with Cubics approach for solving nonconvex optimization problems and propose a new variant based on inexact Hessian information chosen dynamically. The theoretical analysis of the proposed procedure is given. The key property of ARC framework, constituted by optimal worst-case function/derivative evaluation bounds for first- and second-order critical point, is guaranteed. Application to large-scale finite-sum minimization based on subsampled Hessian is discussed and analyzed in both a deterministic and probabilistic manner and equipped with numerical experiments on synthetic and real datasets.