A Note on Inexact Condition for Cubic Regularized Newton's Method

TL;DR

This paper refines the inexact cubic-regularized Newton's method for nonconvex optimization by establishing a practical inexactness condition based solely on current iterates, ensuring the same convergence rate.

Contribution

It introduces an adaptive inexactness condition that depends only on current iterates, fixing previous limitations and providing a new convergence analysis approach.

Findings

Proves convergence rate under the new inexactness condition.

Ensures practical implementability of inexact CR methods.

Controls function value decrease over total iterations.

Abstract

This note considers the inexact cubic-regularized Newton's method (CR), which has been shown in \cite{Cartis2011a} to achieve the same order-level convergence rate to a secondary stationary point as the exact CR \citep{Nesterov2006}. However, the inexactness condition in \cite{Cartis2011a} is not implementable due to its dependence on future iterates variable. This note fixes such an issue by proving the same convergence rate for nonconvex optimization under an inexact adaptive condition that depends on only the current iterate. Our proof controls the sufficient decrease of the function value over the total iterations rather than each iteration as used in the previous studies, which can be of independent interest in other contexts.