Optimizing Adaptive Video Streaming with Human Feedback

TL;DR

This paper introduces Jade, a reinforcement learning framework that incorporates human feedback and relative QoE modeling to optimize adaptive video streaming, significantly enhancing user experience across various network conditions.

Contribution

Jade is the first to integrate human feedback with rank-based QoE models and entropy-aware reinforcement learning for adaptive video streaming optimization.

Findings

Jade improves QoE by up to 38.13% across different network scenarios.

The approach effectively accounts for user heterogeneity in QoE assessment.

Both linear and DNN architectures demonstrate strong performance and generalization.

Abstract

Quality of Experience~(QoE)-driven adaptive bitrate (ABR) algorithms are typically optimized using QoE models that are based on the mean opinion score~(MOS), while such principles may not account for user heterogeneity on rating scales, resulting in unexpected behaviors. In this paper, we propose Jade, which leverages reinforcement learning with human feedback~(RLHF) technologies to better align the users' opinion scores. Jade's rank-based QoE model considers relative values of user ratings to interpret the subjective perception of video sessions. We implement linear-based and Deep Neural Network (DNN)-based architectures for satisfying both accuracy and generalization ability. We further propose entropy-aware reinforced mechanisms for training policies with the integration of the proposed QoE models. Experimental results demonstrate that Jade performs favorably on conventional metrics, such as quality and stall ratio, and improves QoE by 8.09%-38.13% in different network conditions, emphasizing the importance of user heterogeneity in QoE modeling and the potential of combining linear-based and DNN-based models for performance improvement.