Worst-Case Complexity of TRACE with Inexact Subproblem Solutions for Nonconvex Smooth Optimization

TL;DR

This paper extends the TRACE algorithm for nonconvex smooth optimization to allow inexact subproblem solutions, maintaining optimal iteration complexity while improving worst-case Hessian-vector product complexity, and demonstrates its effectiveness through experiments.

Contribution

It introduces an inexact subproblem solution approach for TRACE, preserving optimal complexity and enhancing efficiency for large-scale problems.

Findings

Maintains ${ m O}(\epsilon^{-3/2})$ iteration complexity.

Improves worst-case Hessian-vector product complexity.

Numerical experiments show favorable comparison to state-of-the-art methods.

Abstract

An algorithm for solving nonconvex smooth optimization problems is proposed, analyzed, and tested. The algorithm is an extension of the Trust Region Algorithm with Contractions and Expansions (TRACE) [Math. Prog. 162(1):132, 2017]. In particular, the extension allows the algorithm to use inexact solutions of the arising subproblems, which is an important feature for solving large-scale problems. Inexactness is allowed in a manner such that the optimal iteration complexity of ${\cal O}(\epsilon^{-3/2})$ for attaining an $\epsilon$-approximate first-order stationary point is maintained while the worst-case complexity in terms of Hessian-vector products may be significantly improved as compared to the original TRACE. Numerical experiments show the benefits of allowing inexact subproblem solutions and that the algorithm compares favorably to a state-of-the-art technique.