Convergence of Markov Chains for Constant Step-size Stochastic Gradient Descent with Separable Functions

TL;DR

This paper analyzes the long-term behavior of constant step-size stochastic gradient descent (SGD) on separable functions, revealing a Markov chain structure with invariant measures, convergence properties, and bifurcation phenomena that challenge traditional diffusion approximations.

Contribution

It introduces a Doeblin-type decomposition for SGD Markov chains on separable functions, showing convergence to invariant measures and highlighting complex dynamics like bifurcations.

Findings

Invariant measures form a convex hull and are global attractors.

SGD can leave the global minimum, contradicting diffusion approximation assumptions.

Bifurcations enable transitions between local minima.

Abstract

Stochastic gradient descent (SGD) is a popular algorithm for minimizing objective functions that arise in machine learning. For constant step-sized SGD, the iterates form a Markov chain on a general state space. Focusing on a class of separable (non-convex) objective functions, we establish a "Doeblin-type decomposition," in that the state space decomposes into a uniformly transient set and a disjoint union of absorbing sets. Each of the absorbing sets contains a unique invariant measure, with the set of all invariant measures being the convex hull. Moreover the set of invariant measures are shown to be global attractors to the Markov chain with a geometric convergence rate. The theory is highlighted with examples that show: (1) the failure of the diffusion approximation to characterize the long-time dynamics of SGD; (2) the global minimum of an objective function may lie outside the support of the invariant measures (i.e., even if initialized at the global minimum, SGD iterates will leave); and (3) bifurcations may enable the SGD iterates to transition between two local minima. Key ingredients in the theory involve viewing the SGD dynamics as a monotone iterated function system and establishing a "splitting condition" of Dubins and Freedman 1966 and Bhattacharya and Lee 1988.