A General Framework for Analyzing Stochastic Dynamics in Learning Algorithms

TL;DR

This paper introduces a general, probabilistic framework to analyze stochastic dynamics in learning algorithms, addressing the entanglement of objective and noise, and unifying analysis across diverse problems.

Contribution

It presents a three-step recipe using probability theory to analyze stochastic learning algorithms, enabling unified and improved convergence guarantees.

Findings

Unified analysis for stochastic gradient descent, PCA, and bandit problems

Achieved or improved state-of-the-art convergence bounds

Demonstrated flexibility and power of the framework

Abstract

One of the challenges in analyzing learning algorithms is the circular entanglement between the objective value and the stochastic noise. This is also known as the "chicken and egg" phenomenon and traditionally, there is no principled way to tackle this issue. People solve the problem by utilizing the special structure of the dynamic, and hence the analysis would be difficult to generalize. In this work, we present a streamlined three-step recipe to tackle the "chicken and egg" problem and give a general framework for analyzing stochastic dynamics in learning algorithms. Our framework composes standard techniques from probability theory, such as stopping time and martingale concentration. We demonstrate the power and flexibility of our framework by giving a unifying analysis for three very different learning problems with the last iterate and the strong uniform high probability convergence guarantee. The problems are stochastic gradient descent for strongly convex functions, streaming principal component analysis, and linear bandit with stochastic gradient descent updates. We either improve or match the state-of-the-art bounds on all three dynamics.