On the global convergence of randomized coordinate gradient descent for non-convex optimization

TL;DR

This paper proves that randomized coordinate gradient descent almost surely avoids strict saddle points and converges to local minima in non-convex optimization, under broad assumptions.

Contribution

It provides the first rigorous analysis showing global convergence to local minima for coordinate descent in non-convex problems with random coordinate selection.

Findings

Algorithm almost surely escapes strict saddle points

Converges to local minima under generic conditions

Analysis based on nonlinear random dynamical systems

Abstract

In this work, we analyze the global convergence property of coordinate gradient descent with random choice of coordinates and stepsizes for non-convex optimization problems. Under generic assumptions, we prove that the algorithm iterate will almost surely escape strict saddle points of the objective function. As a result, the algorithm is guaranteed to converge to local minima if all saddle points are strict. Our proof is based on viewing coordinate descent algorithm as a nonlinear random dynamical system and a quantitative finite block analysis of its linearization around saddle points.