Fast Algorithms for Online Stochastic Convex Programming

TL;DR

This paper introduces the online stochastic convex programming problem, providing fast algorithms with near-optimal regret guarantees for various stochastic online problems, and simplifies existing solutions for online packing.

Contribution

It formulates a general online stochastic convex programming framework and develops efficient algorithms with strong theoretical guarantees, connecting primal-dual methods to online learning.

Findings

Achieves near-optimal regret in stochastic models.

Provides a simpler, faster primal-dual algorithm for online packing.

Establishes a connection between primal-dual methods and online learning.

Abstract

We introduce the online stochastic Convex Programming (CP) problem, a very general version of stochastic online problems which allows arbitrary concave objectives and convex feasibility constraints. Many well-studied problems like online stochastic packing and covering, online stochastic matching with concave returns, etc. form a special case of online stochastic CP. We present fast algorithms for these problems, which achieve near-optimal regret guarantees for both the i.i.d. and the random permutation models of stochastic inputs. When applied to the special case online packing, our ideas yield a simpler and faster primal-dual algorithm for this well studied problem, which achieves the optimal competitive ratio. Our techniques make explicit the connection of primal-dual paradigm and online learning to online stochastic CP.