Cutting plane methods can be extended into nonconvex optimization

TL;DR

This paper demonstrates that cutting plane methods can be adapted for nonconvex optimization to achieve faster expected runtimes for finding approximate stationary points, surpassing previous cubic regularized Newton methods.

Contribution

It introduces an extension of cutting plane methods to nonconvex functions, achieving an improved expected runtime of O(ε^{-4/3}) for ε-stationary points.

Findings

Achieves expected runtime of O(ε^{-4/3}) for nonconvex optimization.

Improves upon previous O(ε^{-3/2}) runtime of cubic regularized Newton.

Utilizes the convex until proven guilty principle in the new method.

Abstract

We show that it is possible to obtain an $O(\epsilon^{-4/3})$ expected runtime --- including computational cost --- for finding $\epsilon$-stationary points of smooth nonconvex functions using cutting plane methods. This improves on the best-known epsilon dependence achieved by cubic regularized Newton of $O(\epsilon^{-3/2})$ as proved by Nesterov and Polyak (2006). Our techniques utilize the convex until proven guilty principle proposed by Carmon, Duchi, Hinder, and Sidford (2017).