PL conditions do not guarantee convergence of gradient descent-ascent dynamics

TL;DR

This paper demonstrates that satisfying Polyak-Lojasiewicz conditions alone does not ensure convergence of gradient descent-ascent dynamics to saddle points, highlighting limitations in current convergence guarantees.

Contribution

The authors provide a counterexample showing that PL conditions are insufficient for convergence in gradient descent-ascent methods, even under strong convexity.

Findings

A function satisfying PL conditions can cause gradient descent-ascent to fail to converge.

Convergence guarantees based solely on PL conditions are incomplete.

Strong convexity in one variable does not guarantee convergence in saddle point dynamics.

Abstract

We give an example of a function satisfying a two-sided Polyak-Lojasiewicz condition but for which a gradient descent-ascent flow line fails to converge to the saddle point, circling around it instead. We can even impose the function to be strongly convex in one variable and to satisfy a PL condition in the other variable.