Fast convex optimization via inertial systems with asymptotically vanishing viscosity and Hessian-driven damping

TL;DR

This paper investigates the convergence rates of inertial algorithms derived from a system with vanishing viscosity and Hessian damping, providing insights into accelerated gradient methods.

Contribution

It introduces a family of inertial algorithms with proven sublinear and linear convergence rates under convexity conditions, connecting to Nesterov's method.

Findings

Sublinear convergence for convex functions satisfying Polyak-asiewicz inequality.

Linear convergence for strongly convex functions.

Enhanced understanding of Nesterov's accelerated gradient method.

Abstract

We study the convergence rate of a family of inertial algorithms, which can be obtained by discretization of an inertial system combining asymptotic vanishing viscous and Hessian-driven damping. We establish a fast sublinear convergence rate in case the objective function is convex and satisfies Polyak-\L ojasiewicz inequality. We also establish a linear convergence rate for strongly convex functions. The results can provide more insights into the convergence property of Nesterov's accelerated gradient method.