A Simple Convergence Time Analysis of Drift-Plus-Penalty for Stochastic Optimization and Convex Programs

TL;DR

This paper provides a simplified convergence time analysis for the drift-plus-penalty algorithm, extending its applicability to more general stochastic and convex optimization problems without requiring the Slater condition.

Contribution

It offers a simpler proof of convergence time bounds for the drift-plus-penalty method, applicable to broader problem classes including convex and linear programs.

Findings

Proves $O(1/\epsilon^2)$ convergence time without Slater condition.

Extends analysis to general convex and distributed optimization problems.

Simplifies existing proofs for drift-plus-penalty convergence bounds.

Abstract

This paper considers the problem of minimizing the time average of a stochastic process subject to time average constraints on other processes. A canonical example is minimizing average power in a data network subject to multi-user throughput constraints. Another example is a (static) convex program. Under a Slater condition, the drift-plus-penalty algorithm is known to provide an $O(\epsilon)$ approximation to optimality with a convergence time of $O(1/\epsilon^2)$. This paper proves the same result with a simpler technique and in a more general context that does not require the Slater condition. This paper also emphasizes application to basic convex programs, linear programs, and distributed optimization problems.