Solving convex optimization problems via a second order dynamical system with implicit Hessian damping and Tikhonov regularization

TL;DR

This paper introduces a second order dynamical system with implicit Hessian damping and Tikhonov regularization, leading to inertial algorithms that rapidly converge to the global minimum or minimal norm solution of convex optimization problems.

Contribution

It proposes a novel dynamical system with implicit Hessian damping and Tikhonov regularization, enabling new inertial algorithms with proven convergence properties.

Findings

Objective function value converges rapidly to the global minimum.

Trajectories can converge weakly or strongly depending on regularization parameters.

Velocities tend to zero quickly, ensuring stability.

Abstract

This paper deals with a second order dynamical system with a Tikhonov regularization term in connection to the minimization problem of a convex Fr\'echet differentiable function. The fact that beside the asymptotically vanishing damping we also consider an implicit Hessian driven damping in the dynamical system under study allows us, via straightforward explicit discretization, to obtain inertial algorithms of gradient type. We show that the value of the objective function in a generated trajectory converges rapidly to the global minimum of the objective function and depending the Tikhonov regularization parameter the generated trajectory converges weakly to a minimizer of the objective function or the generated trajectory converges strongly to the element of minimal norm from the $\argmin$ set of the objective function. We also obtain the fast convergence of the velocities towards zero and some integral estimates. Our analysis reveals that the Tikhonov regularization parameter and the damping parameters are strongly correlated, there is a setting of the parameters that separates the cases when weak convergence of the trajectories to a minimizer and strong convergence of the trajectories to the minimal norm minimizer can be obtained.