Stochastic Optimization for Markov Modulated Networks with Application to Delay Constrained Wireless Scheduling

TL;DR

This paper introduces a stochastic optimization-based scheduling algorithm for wireless networks with delay constraints, maximizing throughput and stabilizing queues by solving Markov decision problems.

Contribution

It develops a novel scheduling approach that handles delay constraints in wireless networks by reducing the problem to weighted stochastic shortest path problems.

Findings

Algorithm achieves near-optimal throughput utility.

Stabilizes all queues under delay constraints.

Discusses complexity and delay trade-offs.

Abstract

We consider a wireless system with a small number of delay constrained users and a larger number of users without delay constraints. We develop a scheduling algorithm that reacts to time varying channels and maximizes throughput utility (to within a desired proximity), stabilizes all queues, and satisfies the delay constraints. The problem is solved by reducing the constrained optimization to a set of weighted stochastic shortest path problems, which act as natural generalizations of max-weight policies to Markov decision networks. We also present approximation results for the corresponding shortest path problems, and discuss the additional complexity and delay incurred as compared to systems without delay constraints. The solution technique is general and applies to other constrained stochastic decision problems.