The Fastest Known First-Order Method for Minimizing Twice Continuously Differentiable Smooth Strongly Convex Functions

TL;DR

This paper introduces C2M, a new first-order optimization algorithm that converges faster than the Triple Momentum method for smooth, strongly convex functions that are twice continuously differentiable, without extra computational cost.

Contribution

The paper proposes C2M, a novel optimization algorithm that improves convergence rates over TM for twice continuously differentiable functions, with proven global convergence and no additional cost.

Findings

C2M outperforms TM in convergence speed on twice differentiable functions.

C2M has a strictly faster worst-case convergence rate than TM.

Numerical experiments confirm theoretical advantages of C2M.

Abstract

We consider iterative gradient-based optimization algorithms applied to functions that are smooth and strongly convex. The fastest globally convergent algorithm for this class of functions is the Triple Momentum (TM) method. We show that if the objective function is also twice continuously differentiable, a new, faster algorithm emerges, which we call $C^2$-Momentum (C2M). We prove that C2M is globally convergent and that its worst-case convergence rate is strictly faster than that of TM, with no additional computational cost. We validate our theoretical findings with numerical examples, demonstrating that C2M outperforms TM when the objective function is twice continuously differentiable.