Multimedia Communication Quality Assessment Testbeds

TL;DR

This paper compares multimedia quality assessment testbeds using VLC and GStreamer frameworks, demonstrating that GStreamer provides more robust streaming and more accurate quality metrics, leading to improved quality estimation models.

Contribution

The paper introduces a GStreamer-based multimedia pipeline that enhances robustness and accuracy in quality assessment, surpassing previous VLC-based methods.

Findings

GStreamer pipeline is more robust to network degradations.

RTCP statistics from GStreamer are more accurate.

Better quality estimation models were developed using improved data.

Abstract

We make an intensive use of multimedia frameworks in our research on modeling the perceived quality estimation in streaming services and real-time communications. In our preliminary work, we have used the VLC VOD software to generate reference audiovisual files with various degree of coding and network degradations. We have successfully built machine learning based models on the subjective quality dataset we have generated using these files. However, imperfections in the dataset introduced by the multimedia framework we have used prevented us from achieving the full potential of these models. In order to develop better models, we have re-created our end-to-end multimedia pipeline using the GStreamer framework for audio and video streaming. A GStreamer based pipeline proved to be significantly more robust to network degradations than the VLC VOD framework and allowed us to stream a video flow at a loss rate up to 5\% packet very easily. GStreamer has also enabled us to collect the relevant RTCP statistics that proved to be more accurate than network-deduced information. This dataset is free to the public. The accuracy of the statistics eventually helped us to generate better performing perceived quality estimation models. In this paper, we present the implementation of these VLC and GStreamer-based multimedia communication quality assessment testbeds with the references to their publicly available code bases.