Randomized coordinate gradient descent almost surely escapes strict saddle points

TL;DR

This paper proves that randomized coordinate gradient descent almost surely avoids strict saddle points in nonconvex optimization by analyzing its dynamics as a nonlinear random system.

Contribution

It introduces a novel analysis framework using the center-stable manifold theorem to show almost sure escape from strict saddle points.

Findings

Randomized coordinate gradient descent almost surely escapes strict saddle points.

The analysis uses nonlinear random dynamical systems and the center-stable manifold theorem.

Provides theoretical guarantees for the behavior of the algorithm in nonconvex landscapes.

Abstract

We analyze the behavior of randomized coordinate gradient descent for nonconvex optimization, proving that under standard assumptions, the iterates almost surely escape strict saddle points. By formulating the method as a nonlinear random dynamical system and characterizing neighborhoods of critical points, we establish this result through the center-stable manifold theorem.