Fast second-order dynamics with slow vanishing damping approaching the zeros of a monotone and continuous operator

TL;DR

This paper introduces a second-order dynamical system with slow vanishing damping to efficiently find zeros of monotone operators, achieving fast convergence rates and weak trajectory convergence.

Contribution

It develops a novel second-order system with a damping term of order 1/t^r and a growth condition for beta(t), providing new convergence rates and trajectory behavior analysis.

Findings

Achieves o(1/(t^{2r} beta(t))) convergence rates for operator norms and gap functions.

Allows exponential growth of beta(t) for linear convergence when r<1.

Demonstrates weak convergence of trajectories to zeros and primal-dual solutions.

Abstract

In this work, we approach the problem of finding the zeros of a continuous and monotone operator through a second-order dynamical system with a damping term of the form $1/t^{r}$, where $r\in [0, 1]$. The system features the time derivative of the operator evaluated along the trajectory, which is a Hessian-driven type damping term when the governing operator comes from a potential. Also entering the system is a time rescaling parameter $\beta(t)$ which satisfies a certain growth condition. We derive $o\left(\frac{1}{t^{2r}\beta(t)}\right)$ convergence rates for the norm of the operator evaluated along the generated trajectories as well as for a gap function which serves as a measure of optimality for the associated variational inequality. The parameter $r$ enters the growth condition for $\beta(t)$: when $r < 1$, the damping $1/t^{r}$ approaches zero at a slower speed than Nesterov's $1/t$ damping; in this case, we are allowed to choose $\beta(t)$ to be an exponential function, thus having linear convergence rates for the involved quantities. We also show weak convergence of the trajectories towards zeros of the governing operator. Through a particular choice for the operator, we establish a connection with the problem of minimizing a smooth and convex function with linear constraints. The convergence rates we derived in the operator case are inherited by the objective function evaluated at the trajectories and for the feasibility gap. We also prove weak convergence of the trajectories towards primal-dual solutions of the problem. A discretization of the dynamical system yields an implicit algorithm that exhibits analogous convergence properties to its continuous counterpart. We complement our theoretical findings with two numerical experiments.