# Active-set Newton-MR methods for nonconvex optimization problems with bound constraints

## TL;DR

This paper develops active-set Newton-MR methods for nonconvex bound-constrained optimization, providing theoretical guarantees and numerical comparisons showing efficiency improvements over existing methods.

## Contribution

It introduces novel active-set methods combining MINRES and cubic regularization, with proven complexity bounds for finding approximate stationary points.

## Key findings

- SPG-based method requires O(nε^{-2}) oracle calls for stationary points.
- Cubic regularization method requires O(n|log₂(ε)|ε^{-3/2}) oracle calls.
- Numerical experiments show benefits of MINRES over CG in practice.

## Abstract

This paper presents active-set methods for minimizing nonconvex twice-continuously differentiable functions subject to bound constraints. Within the faces of the feasible set, we employ descent methods with Armijo line search, utilizing approximated Newton directions obtained through the Minimum Residual (MINRES) method. To escape the faces, we investigate the use of the Spectral Projected Gradient (SPG) method and a tailored variant of the Cubic Regularization of Newton's method for bound-constrained problems. We provide theoretical guarantees, demonstrating that when the objective function has a Lipschitz continuous gradient, the SPG-based method requires no more than $\mathcal{O}(n\epsilon^{-2})$ oracle calls to find $\epsilon$-approximate stationary points, where $n$ is the problem dimension. Furthermore, if the objective function also has a Lipschitz continuous Hessian, we show that the method based on cubic regularization requires no more than $\mathcal{O}\left(n|\log_{2}(\epsilon)|\epsilon^{-3/2}\right)$ oracle calls to achieve the same goal. We emphasize that, under certain hypotheses, the method achieves $O(\epsilon^{3/2})$ descent within the faces without resorting to cubic regularization. Numerical experiments are conducted to compare the proposed methods with existing active-set methods, highlighting the potential benefits of using MINRES instead of the Conjugate Gradient (CG) method for approximating Newton directions.