Heavy-ball dynamics with Hessian-driven damping for non-convex optimization under the {\L}ojasiewicz condition

TL;DR

This paper analyzes the convergence of heavy-ball dynamics with Hessian-driven damping in non-convex optimization under the { extbackslash}Lojasiewicz condition, providing optimal rates and stability insights.

Contribution

It offers the first tight convergence rate guarantees for this dynamics in non-convex settings, improving previous results and suggesting new damping parameter tuning.

Findings

Linear convergence rates depend on damping and { extbackslash}Lojasiewicz exponent.

Dynamics avoid strict saddle points from almost all initial conditions.

Stability estimates in the presence of perturbations and inexact gradients.

Abstract

In this paper, we examine the convergence properties of heavy-ball dynamics with Hessian-driven damping in smooth non-convex optimization problems satisfying a {\L}ojasiewicz condition. In this general setting, we provide a series of tight, worst-case optimal convergence rate guarantees as a function of the dynamics' friction coefficients and the {\L}ojasiewicz exponent of the problem's objective function. Importantly, the linear rates that we obtain improve on previous available rates and they suggest a different tuning of the dynamics' damping terms, even in the strongly convex regime. We complement our analysis with a range of stability estimates in the presence of perturbation errors and inexact gradient input, as well as an avoidance result showing that the dynamics under study avoid strict saddle points from almost every initial condition,