A Globally Convergent Third-Order Newton Method via Unified Semidefinite Programming Subproblems

TL;DR

This paper introduces ALMTON, a globally convergent third-order Newton method for unconstrained nonconvex optimization that uses semidefinite programming subproblems, improving convergence and efficiency over classical methods.

Contribution

The paper presents ALMTON, a novel third-order Newton method employing adaptive Levenberg-Marquardt regularization with SDP subproblems, achieving global convergence and better practical performance.

Findings

ALMTON enlarges the basin of attraction compared to classical methods.

It converges more consistently and often in fewer iterations than AR3-interp.

Numerical experiments demonstrate improved progress on challenging landscapes.

Abstract

We propose the Adaptive Levenberg-Marquardt Third-Order Newton Method (ALMTON) for unconstrained nonconvex optimization, providing the first globally convergent realization of the unregularized third-order Newton method. Unlike the standard Adaptive Regularization framework with third-order models (AR3), which enforces global behavior through a quartic term, ALMTON employs an adaptive Levenberg-Marquardt (quadratic) regularization. This choice preserves a cubic model at every iteration, so that every subproblem is a tractable semidefinite program (SDP). In particular, the ALMTON-Simple variant requires exactly one SDP solve per iteration, making the per-iteration cost uniform and predictable. Algorithmically, ALMTON follows a mixed-mode strategy: it attempts an unregularized third-order step whenever the cubic Taylor model admits a strict local minimizer with adequate curvature, and activates (or increases) quadratic regularization only when needed to ensure that the model is well posed and the step is globally reliable. We establish global convergence and prove an $O(\epsilon^{-2})$ worst-case evaluation complexity for computing an $\epsilon$-approximate first-order stationary point. Numerical experiments show that ALMTON enlarges the basin of attraction relative to classical baselines (gradient descent and damped Newton), and can progress on landscapes where second-order methods typically stagnate. When compared with a state-of-the-art third-order implementation (AR3-interp), ALMTON converges more consistently and often in fewer iterations. We also characterize the practical scalability limits of the approach, highlighting the computational bottlenecks introduced by current SDP solvers as dimension grows.