Convergence and Inference of Stream SGD, with Applications to Queueing Systems and Inventory Control

TL;DR

This paper develops a comprehensive analysis framework for stream SGD in online optimization, establishing convergence rates, inference methods, and applications to queueing and inventory systems, advancing efficiency and understanding in operations research.

Contribution

It introduces a systematic framework for analyzing stream SGD, including convergence, regret, inference, and a novel divergence, with applications to queueing and inventory management.

Findings

Proves optimal O(1/T) convergence rates for stream SGD.

Establishes O(log T) regret bounds.

Demonstrates practical applications in queueing and inventory systems.

Abstract

Stream stochastic gradient descent (SGD) is a simple and efficient method for solving online optimization problems in operations research (OR), where data is generated by parameter-dependent Markov chains. Unlike traditional approaches which require increasing batch sizes during iterations, stream SGD uses a single sample per iteration, significantly improving sample efficiency. This paper establishes a systematic framework for analyzing stream SGD, leveraging the Poisson equation solution to address gradient bias and statistical dependence. We prove optimal O(1/T) convergence rates and the state-of-the-art O(log T) regret, while also introducing an online inference method for uncertainty quantification and supporting it by a novel functional central limit theorem. We propose a novel Wasserstein-type divergence to describe the framework's conditions, which makes the assumptions in question directly verified via coupling techniques tailored to underlying OR models. We consider applications in queueing systems and inventory management, demonstrating the practicality and broad relevance, as well as providing new insights into the effectiveness of stream SGD in OR fields.