Nonlinearly preconditioned gradient flows

TL;DR

This paper analyzes a class of nonlinear preconditioned gradient flows, establishing their existence, convergence properties, and connections to mirror descent and optimal control, thus clarifying their structure in non-Euclidean optimization.

Contribution

It introduces a unified analysis of nonlinear preconditioned gradient flows, proving convergence, duality with mirror descent, and links to optimal control problems.

Findings

Global solutions exist under mild assumptions.

Convex costs exhibit sublinear decay in a specific geometry.

Under gradient-dominance, exponential convergence is achieved.

Abstract

We study a continuous-time dynamical system which arises as the limit of a broad class of nonlinearly preconditioned gradient methods. Under mild assumptions, we establish existence of global solutions and derive Lyapunov-based convergence guarantees. For convex costs, we prove a sublinear decay in a geometry induced by some reference function, and under a generalized gradient-dominance condition we obtain exponential convergence. We further uncover a duality connection with mirror descent, and use it to establish that the flow of interest solves an infinite-horizon optimal-control problem of which the value function is the Bregman divergence generated by the cost. These results clarify the structure and optimization behavior of nonlinearly preconditioned gradient flows and connect them to known continuous-time models in non-Euclidean optimization.