Realistic Video Sequences for Subjective QoE Analysis

TL;DR

This paper introduces DashReStreamer, an open-source framework and dataset for creating realistic, adaptively streamed videos to improve subjective QoE assessment in multimedia streaming over the Internet.

Contribution

The paper presents DashReStreamer and a dataset of 234 realistic video clips, enabling more accurate subjective QoE evaluations under real network conditions.

Findings

Created 234 realistic video clips from real network logs

Provided a dataset with network bandwidth profiles and HAS decisions

Facilitated more accurate subjective QoE assessments

Abstract

Multimedia streaming over the Internet (live and on demand) is the cornerstone of modern Internet carrying more than 60% of all traffic. With such high demand, delivering outstanding user experience is a crucial and challenging task. To evaluate user QoE many researchers deploy subjective quality assessments where participants watch and rate videos artificially infused with various temporal and spatial impairments. To aid current efforts in bridging the gap between the mapping of objective video QoE metrics to user experience, we developed DashReStreamer, an open-source framework for re-creating adaptively streamed video in real networks. DashReStreamer utilises a log created by a HAS algorithm run in an uncontrolled environment (i.e., wired or wireless networks), encoding visual changes and stall events in one video file. These videos are applicable for subjective QoE evaluation mimicking realistic network conditions. To supplement DashReStreamer, we re-create 234 realistic video clips, based on video logs collected from real mobile and wireless networks. In addition our dataset contains both video logs with all decisions made by the HASalgorithm and network bandwidth profile illustrating throughput distribution. We believe this dataset and framework will permit other researchers in their pursuit for the final frontier in understanding the impact of video QoE dynamics.