Levenberg-Marquardt dynamics associated to variational inequalities

TL;DR

This paper studies the long-term behavior of trajectories generated by a nonautonomous Levenberg-Marquardt dynamical system in convex optimization, proving convergence under certain conditions.

Contribution

It introduces a new analysis of Levenberg-Marquardt dynamics for variational inequalities, establishing convergence results with specific conditions on parameters.

Findings

Weak convergence to optimal solutions

Convergence of objective function values

Strong convergence under strong convexity

Abstract

In connection with the optimization problem $$\inf_{x\in argmin \Psi}\{\Phi(x)+\Theta(x)\},$$ where $\Phi$ is a proper, convex and lower semicontinuous function and $\Theta$ and $\Psi$ are convex and smooth functions defined on a real Hilbert space, we investigate the asymptotic behavior of the trajectories of the nonautonomous Levenberg-Marquardt dynamical system \begin{equation*}\left\{ \begin{array}{ll} v(t)\in\partial\Phi(x(t))\\ \lambda(t)\dot x(t) + \dot v(t) + v(t) + \nabla \Theta(x(t))+\beta(t)\nabla \Psi(x(t))=0, \end{array}\right.\end{equation*} where $\lambda$ and $\beta$ are functions of time controlling the velocity and the penalty term, respectively. We show weak convergence of the generated trajectory to an optimal solution as well as convergence of the objective function values along the trajectories, provided $\lambda$ is monotonically decreasing, $\beta$ satisfies a growth condition and a relation expressed via the Fenchel conjugate of $\Psi$ is fulfilled. When the objective function is assumed to be strongly convex, we can even show strong convergence of the trajectories.