A Probably Approximately Correct Answer to Distributed Stochastic Optimization in a Non-stationary Environment

TL;DR

This paper introduces an approximate Drift-plus-penalty algorithm for distributed stochastic optimization in non-stationary environments, achieving near-optimal cost with high probability despite delays and evolving system states.

Contribution

It presents a novel method that adapts to non-i.i.d. and non-stationary system states, providing high probability guarantees and improved error bounds.

Findings

Achieves cost within epsilon of optimal with high probability.

Provides a condition on waiting time based on system parameters.

Error bound dependency on sample size w is exponential, improving over previous work.

Abstract

This paper considers a distributed stochastic optimization problem where the goal is to minimize the time average of a cost function subject to a set of constraints on the time averages of a related stochastic processes called penalties. We assume that a delayed information about an event in the system is available as a common information at every user, and the state of the system is evolving in an independent and non-stationary fashion. We show that an approximate Drift-plus-penalty (DPP) algorithm that we propose achieves a time average cost that is within some positive constant epsilon of the optimal cost with high probability. Further, we provide a condition on the waiting time for this result to hold. The condition is shown to be a function of the mixing coefficient, the number of samples (w) used to compute an estimate of the distribution of the state, and the delay. Unlike the existing work, the method used in the paper can be adapted to prove high probability results when the state is evolving in a non-i.i.d and non-stationary fashion. Under mild conditions, we show that the dependency of the error bound on w is exponential, which is a significant improvement compared to the exiting work.