A speed restart scheme for a dynamics with Hessian driven damping

TL;DR

This paper introduces a speed restart scheme for a convex dynamics with Hessian-driven damping, proving linear convergence under quadratic growth and showing improved performance through numerical experiments.

Contribution

It proposes a novel speed restarting scheme for a Hessian-driven damping dynamical system and establishes linear convergence for functions with quadratic growth.

Findings

Linear convergence rate for quadratic growth functions

Numerical evidence of improved performance with restart scheme

Enhanced results for strongly convex functions with Hessian damping

Abstract

In this paper, we analyze a speed restarting scheme for the dynamical system given by $$ \ddot{x}(t) + \dfrac{\alpha}{t}\dot{x}(t) + \nabla \phi(x(t)) + \beta \nabla^2 \phi(x(t))\dot{x}(t)=0, $$ where $\alpha$ and $\beta$ are positive parameters, and $\phi:\mathbb{R}^n \to \mathbb{R}$ is a smooth convex function. If $\phi$ has quadratic growth, we establish a linear convergence rate for the function values along the restarted trajectories. As a byproduct, we improve the results obtained by Su, Boyd and Cand\`es \cite{JMLR:v17:15-084}, obtained in the strongly convex case for $\alpha=3$ and $\beta=0$. Preliminary numerical experiments suggest that both adding a positive Hessian driven damping parameter $\beta$, and implementing the restart scheme help improve the performance of the dynamics and corresponding iterative algorithms as means to approximate minimizers of $\phi$.