Non-Asymptotic Analysis of Stochastic Approximation Algorithms for Streaming Data

TL;DR

This paper develops a non-asymptotic analysis framework for stochastic approximation algorithms in streaming data settings, showing how to accelerate convergence and reduce variance using time-varying mini-batches and averaging techniques.

Contribution

It introduces a novel streaming framework with non-asymptotic convergence rates and demonstrates how to optimize learning rates and averaging for faster, more accurate online learning.

Findings

Polyak-Ruppert averaging achieves optimal convergence bounds.

Time-varying mini-batches combined with averaging improve variance reduction.

Accelerated convergence through adaptive learning rate selection.

Abstract

We introduce a streaming framework for analyzing stochastic approximation/optimization problems. This streaming framework is analogous to solving optimization problems using time-varying mini-batches that arrive sequentially. We provide non-asymptotic convergence rates of various gradient-based algorithms; this includes the famous Stochastic Gradient (SG) descent (a.k.a. Robbins-Monro algorithm), mini-batch SG and time-varying mini-batch SG algorithms, as well as their iterated averages (a.k.a. Polyak-Ruppert averaging). We show i) how to accelerate convergence by choosing the learning rate according to the time-varying mini-batches, ii) that Polyak-Ruppert averaging achieves optimal convergence in terms of attaining the Cramer-Rao lower bound, and iii) how time-varying mini-batches together with Polyak-Ruppert averaging can provide variance reduction and accelerate convergence simultaneously, which is advantageous for many learning problems, such as online, sequential, and large-scale learning. We further demonstrate these favorable effects for various time-varying mini-batches.