A Markov Decision Model for Adaptive Scheduling of Stored Scalable Videos

TL;DR

This paper introduces two scheduling algorithms for stored scalable videos, leveraging a Markov Decision Process framework to optimize video quality over dynamic channels, with one being near-optimal and the other practical.

Contribution

The paper develops a Markov Decision Process-based scheduling algorithm and an online heuristic for scalable video transmission, improving quality optimization under channel variability.

Findings

Both algorithms perform close to the derived upper bound.

The MDP-based policy minimizes mean square error effectively.

The online algorithm is practical and nearly as effective as the optimal policy.

Abstract

We propose two scheduling algorithms that seek to optimize the quality of scalably coded videos that have been stored at a video server before transmission.} The first scheduling algorithm is derived from a Markov Decision Process (MDP) formulation developed here. We model the dynamics of the channel as a Markov chain and reduce the problem of dynamic video scheduling to a tractable Markov decision problem over a finite state space. Based on the MDP formulation, a near-optimal scheduling policy is computed that minimize the mean square error. Using insights taken from the development of the optimal MDP-based scheduling policy, the second proposed scheduling algorithm is an online scheduling method that only requires easily measurable knowledge of the channel dynamics, and is thus viable in practice. Simulation results show that the performance of both scheduling algorithms is close to a performance upper bound also derived in this paper.