FairStream: Fair Multimedia Streaming Benchmark for Reinforcement Learning Agents

TL;DR

This paper introduces FairStream, a comprehensive multi-agent benchmark for fair multimedia streaming using reinforcement learning, highlighting the challenges and analyzing baseline approaches across diverse traffic scenarios.

Contribution

It presents a novel multi-agent environment for fair multimedia streaming, addressing key challenges and providing baseline analyses that reveal limitations of common RL algorithms.

Findings

PPO algorithm is outperformed by a simple greedy heuristic.

Baseline approaches reveal challenges in fairness and efficiency.

Environment covers diverse traffic classes and realistic streaming challenges.

Abstract

Multimedia streaming accounts for the majority of traffic in today's internet. Mechanisms like adaptive bitrate streaming control the bitrate of a stream based on the estimated bandwidth, ideally resulting in smooth playback and a good Quality of Experience (QoE). However, selecting the optimal bitrate is challenging under volatile network conditions. This motivated researchers to train Reinforcement Learning (RL) agents for multimedia streaming. The considered training environments are often simplified, leading to promising results with limited applicability. Additionally, the QoE fairness across multiple streams is seldom considered by recent RL approaches. With this work, we propose a novel multi-agent environment that comprises multiple challenges of fair multimedia streaming: partial observability, multiple objectives, agent heterogeneity and asynchronicity. We provide and analyze baseline approaches across five different traffic classes to gain detailed insights into the behavior of the considered agents, and show that the commonly used Proximal Policy Optimization (PPO) algorithm is outperformed by a simple greedy heuristic. Future work includes the adaptation of multi-agent RL algorithms and further expansions of the environment.