Proximal Gradient Dynamics: Monotonicity, Exponential Convergence, and Applications

TL;DR

This paper analyzes the proximal gradient dynamics, establishing monotonicity and exponential convergence properties, and extends these results to time-varying problems with applications to LASSO, matrix problems, and neural networks.

Contribution

It introduces a new condition ensuring exponential convergence and links it to the proximal Polyak-Łojasiewicz condition, extending analysis to time-varying optimization.

Findings

Cost function decreases monotonically along trajectories.

New condition guarantees exponential convergence.

Extensions to time-varying problems with practical applications.

Abstract

In this letter we study the proximal gradient dynamics. This recently-proposed continuous-time dynamics solves optimization problems whose cost functions are separable into a nonsmooth convex and a smooth component. First, we show that the cost function decreases monotonically along the trajectories of the proximal gradient dynamics. We then introduce a new condition that guarantees exponential convergence of the cost function to its optimal value, and show that this condition implies the proximal Polyak-{\L}ojasiewicz condition. We also show that the proximal Polyak-{\L}ojasiewicz condition guarantees exponential convergence of the cost function. Moreover, we extend these results to time-varying optimization problems, providing bounds for equilibrium tracking. Finally, we discuss applications of these findings, including the LASSO problem, certain matrix based problems and a numerical experiment on a feed-forward neural network.