Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming

TL;DR

This study introduces a new live video streaming QoE dataset, evaluates existing models, and proposes Tao-QoE, a novel end-to-end model for assessing live video quality based on semantic and motion features.

Contribution

The paper presents the first live video QoE dataset, benchmarks existing models, and introduces Tao-QoE, a new model that improves QoE prediction accuracy for live streaming.

Findings

Current QoE models struggle with live streaming content.

Tao-QoE outperforms existing models in predicting live video QoE.

The TaoLive QoE dataset includes diverse streaming distortions.

Abstract

In recent years, live video streaming has gained widespread popularity across various social media platforms. Quality of experience (QoE), which reflects end-users' satisfaction and overall experience, plays a critical role for media service providers to optimize large-scale live compression and transmission strategies to achieve perceptually optimal rate-distortion trade-off. Although many QoE metrics for video-on-demand (VoD) have been proposed, there remain significant challenges in developing QoE metrics for live video streaming. To bridge this gap, we conduct a comprehensive study of subjective and objective QoE evaluations for live video streaming. For the subjective QoE study, we introduce the first live video streaming QoE dataset, TaoLive QoE, which consists of $42$ source videos collected from real live broadcasts and $1,155$ corresponding distorted ones degraded due to a variety of streaming distortions, including conventional streaming distortions such as compression, stalling, as well as live streaming-specific distortions like frame skipping, variable frame rate, etc. Subsequently, a human study was conducted to derive subjective QoE scores of videos in the TaoLive QoE dataset. For the objective QoE study, we benchmark existing QoE models on the TaoLive QoE dataset as well as publicly available QoE datasets for VoD scenarios, highlighting that current models struggle to accurately assess video QoE, particularly for live content. Hence, we propose an end-to-end QoE evaluation model, Tao-QoE, which integrates multi-scale semantic features and optical flow-based motion features to predicting a retrospective QoE score, eliminating reliance on statistical quality of service (QoS) features.