Parameter-Free Accelerated Quasi-Newton Method for Nonconvex Optimization

TL;DR

This paper introduces a parameter-free quasi-Newton method for nonconvex optimization that achieves near-optimal complexity bounds without prior knowledge of problem parameters, combining acceleration, regularization, and a scaled PSB update.

Contribution

It presents a novel parameter-free quasi-Newton algorithm for nonconvex optimization with improved complexity bounds and no need for prior parameter tuning.

Findings

Achieves $ ilde{O}(d^{1/4} ext{ε}^{-13/8})$ gradient evaluations to find an ε-stationary point.

Does not require prior knowledge of Lipschitz constants or target accuracy.

Combines momentum acceleration, quartic regularization, and a scaled PSB update.

Abstract

We propose a quasi-Newton-type method for nonconvex optimization with Lipschitz continuous gradients and Hessians. The algorithm finds an $\varepsilon$-stationary point within $\tilde{\mathrm{O}}(d^{1/4} \varepsilon^{-13/8})$ gradient evaluations, where $d$ is the problem dimension. Although this bound includes an additional logarithmic factor compared with the best known complexity, our method is parameter-free in the sense that it requires no prior knowledge of problem-dependent parameters such as Lipschitz constants or the optimal value. Moreover, it does not need the target accuracy $\varepsilon$ or the total number of iterations to be specified in advance. The result is achieved by combining several key ideas: momentum-based acceleration, quartic regularization for subproblems, and a scaled variant of the Powell-symmetric-Broyden (PSB) update.