Complexity of a Projected Newton-CG Method for Optimization with Bounds

TL;DR

This paper introduces a new projected Newton-CG method for bound-constrained nonconvex optimization, providing theoretical complexity guarantees and demonstrating practical effectiveness through computational experiments.

Contribution

It combines classical gradient projection with Newton-CG techniques and introduces a new approximate second-order optimality definition, deriving complexity bounds for the method.

Findings

Method achieves good worst-case complexity guarantees.

Computational results show practical efficiency on low-rank matrix problems.

Provides a new framework for analyzing nonconvex bound-constrained optimization.

Abstract

This paper describes a method for solving smooth nonconvex minimization problems subject to bound constraints with good worst-case complexity guarantees and practical performance. The method contains elements of two existing methods: the classical gradient projection approach for bound-constrained optimization and a recently proposed Newton-conjugate gradient algorithm for unconstrained nonconvex optimization. Using a new definition of approximate second-order optimality parametrized by some tolerance $\epsilon$ (which is compared with related definitions from previous works), we derive complexity bounds in terms of $\epsilon$ for both the number of iterations required and the total amount of computation. The latter is measured by the number of gradient evaluations or Hessian-vector products. We also describe illustrative computational results on several test problems from low-rank matrix optimization.