A fast continuous time approach for non-smooth convex optimization with time scaling and Tikhonov regularization

TL;DR

This paper introduces a second-order differential equation with time scaling and Tikhonov regularization for non-smooth convex optimization, demonstrating improved convergence properties and strong convergence to minimal norm solutions.

Contribution

It proposes a novel dynamical system combining viscous and Hessian-driven damping with time scaling and Tikhonov regularization, enhancing convergence in non-smooth convex optimization.

Findings

Preserves and improves convergence rates of the function and Moreau envelope

Proves strong convergence of trajectories to minimal norm solutions

Provides numerical results validating theoretical claims

Abstract

In a Hilbert setting we aim to study a second order in time differential equation, combining viscous and Hessian-driven damping, containing a time scaling parameter function and a Tikhonov regularization term. The dynamical system is related to the problem of minimization of a nonsmooth convex function. In the formulation of the problem as well as in our analysis we use the Moreau envelope of the objective function and its gradient and heavily rely on their properties. We show that there is a setting where the newly introduced system preserves and even improves the well-known fast convergence properties of the function and Moreau envelope along the trajectories and also of the gradient of Moreau envelope due to the presence of time scaling. Moreover, in a different setting we prove strong convergence of the trajectories to the element of the minimal norm from the set of all minimizers of the objective. The manuscript concludes with various numerical results.