Time-Average Stochastic Optimization with Non-convex Decision Set and its Convergence

TL;DR

This paper analyzes the convergence behavior of time-average stochastic optimization with non-convex decision sets, revealing phases and conditions for improved convergence times, applicable in networking and operations research.

Contribution

It demonstrates that Lyapunov optimization exhibits distinct transient and steady state phases, and provides improved convergence time bounds under certain assumptions.

Findings

Convergence time is $O(1/\epsilon)$ with a unique Lagrange multiplier.

Convergence time is $O(1/\epsilon^{1.5})$ under non-polyhedral conditions.

Simulations suggest broader applicability beyond the assumptions.

Abstract

This paper considers time-average stochastic optimization, where a time average decision vector, an average of decision vectors chosen in every time step from a time-varying (possibly non-convex) set, minimizes a convex objective function and satisfies convex constraints. This formulation has applications in networking and operations research. In general, time-average stochastic optimization can be solved by a Lyapunov optimization technique. This paper shows that the technique exhibits a transient phase and a steady state phase. When the problem has a unique vector of Lagrange multipliers, the convergence time can be improved. By starting the time average in the steady state the convergence times become $O(1/\epsilon)$ under a locally-polyhedral assumption and $O(1/\epsilon^{1.5})$ under a locally-non-polyhedral assumption, where $\epsilon$ denotes the proximity to the optimal objective cost. Simulations suggest that the results may hold more generally without the unique Lagrange multiplier assumption.