Fast convex optimization via a third-order in time evolution equation: TOGES-V an improved version of TOGES

TL;DR

This paper introduces an improved third-order in time evolution equation for convex optimization, achieving faster convergence rates and reduced oscillations, with special results for strongly convex functions and extensions to nonsmooth cases.

Contribution

It develops a new third-order dynamic system with enhanced convergence properties and damping mechanisms, extending previous work in convex optimization dynamics.

Findings

Convergence rate of 1/t^3 for function values when damping parameter > 3.

Introduction of Hessian-driven damping reduces oscillations.

Exponential convergence for strongly convex functions.

Abstract

In a Hilbert space setting H, for convex optimization, we analyze the fast convergence properties as t tends to infinity of the trajectories generated by a third-order in time evolution system. The function f to minimize is supposed to be convex, continuously differentiable, with a nonempty set of minimizers. It enters into the dynamic through its gradient. Based on this new dynamical system, we improve the results obtained by [Attouch, Chbani, Riahi: Fast convex optimization via a third-order in time evolution equation, Optimization 2020]. As a main result, when the damping parameter $\alpha$ satisfies $\alpha > 3$, we show that the convergence of the values at the order 1/t3 as t goes to infinity, as well as the convergence of the trajectories. We complement these results by introducing into the dynamic an Hessian driven damping term, which reduces the oscillations. In the case of a strongly convex function f, we show an autonomous evolution system of the third order in time with an exponential rate of convergence. All these results have natural extensions to the case of a convex lower semicontinuous function with extended real values. Just replace f with its Moreau envelope.