Fast Convergence of an Inertial Gradient-like System with Vanishing Viscosity

TL;DR

This paper analyzes the convergence behavior of an inertial gradient system with vanishing viscosity in Hilbert spaces, establishing weak and strong convergence results and connecting continuous dynamics to fast convex optimization algorithms.

Contribution

It provides new convergence results for an inertial system with vanishing damping, extending the understanding of accelerated methods in convex optimization.

Findings

Weak convergence to minimizers for ta > 3

Strong convergence in practical cases

Connection to fast convex optimization algorithms

Abstract

In a real Hilbert space $\mathcal H$, we study the fast convergence properties as $t \to + \infty$ of the trajectories of the second-order evolution equation $$ \ddot{x}(t) + \frac{\alpha}{t} \dot{x}(t) + \nabla \Phi (x(t)) = 0, $$ where $\nabla \Phi$ is the gradient of a convex continuously differentiable function $\Phi : \mathcal H \rightarrow \mathbb R$, and $\alpha$ is a positive parameter. In this inertial system, the viscous damping coefficient $\frac{\alpha}{t}$ vanishes asymptotically in a moderate way. For $\alpha > 3$, we show that any trajectory converges weakly to a minimizer of $\Phi$, just assuming that the set of minimizers is nonempty. The strong convergence is established in various practical situations. These results complement the $\mathcal O(t^{-2})$ rate of convergence for the values obtained by Su, Boyd and Cand\`es. Time discretization of this system, and some of its variants, provides new fast converging algorithms, expanding the field of rapid methods for structured convex minimization introduced by Nesterov, and further developed by Beck and Teboulle. This study also complements recent advances due to Chambolle and Dossal.