A Deep Graph Reinforcement Learning Model for Improving User Experience in Live Video Streaming

TL;DR

This paper introduces a deep graph reinforcement learning model that predicts and enhances user experience during live video streams, significantly increasing high-quality viewers early in the event.

Contribution

The paper proposes a novel deep graph reinforcement learning approach tailored for live video streaming, incorporating a gradient boosting strategy for global model training across diverse events.

Findings

Model outperforms baseline strategies in real-world datasets.

Significantly increases high-quality viewers by at least 75% early in streams.

Effective in diverse live streaming scenarios.

Abstract

In this paper we present a deep graph reinforcement learning model to predict and improve the user experience during a live video streaming event, orchestrated by an agent/tracker. We first formulate the user experience prediction problem as a classification task, accounting for the fact that most of the viewers at the beginning of an event have poor quality of experience due to low-bandwidth connections and limited interactions with the tracker. In our model we consider different factors that influence the quality of user experience and train the proposed model on diverse state-action transitions when viewers interact with the tracker. In addition, provided that past events have various user experience characteristics we follow a gradient boosting strategy to compute a global model that learns from different events. Our experiments with three real-world datasets of live video streaming events demonstrate the superiority of the proposed model against several baseline strategies. Moreover, as the majority of the viewers at the beginning of an event has poor experience, we show that our model can significantly increase the number of viewers with high quality experience by at least 75% over the first streaming minutes. Our evaluation datasets and implementation are publicly available at https://publicresearch.z13.web.core.windows.net