Towards Reduced Reference Parametric Models for Estimating Audiovisual Quality in Multimedia Services

TL;DR

This paper presents reduced reference parametric models using machine learning to accurately estimate perceived audiovisual quality in multimedia services, leveraging network parameters and side information.

Contribution

The study introduces novel reduced reference models based on Random Forest and Neural Networks for audiovisual quality estimation, emphasizing the effectiveness of side information.

Findings

Random Forest models outperform Neural Networks in accuracy.

Side information significantly improves model performance.

Models estimate perceived quality well using network parameters.

Abstract

We have developed reduced reference parametric models for estimating perceived quality in audiovisual multimedia services. We have created 144 unique configurations for audiovisual content including various application and network parameters such as bitrates and distortions in terms of bandwidth, packet loss rate and jitter. To generate the data needed for model training and validation we have tasked 24 subjects, in a controlled environment, to rate the overall audiovisual quality on the absolute category rating (ACR) 5-level quality scale. We have developed models using Random Forest and Neural Network based machine learning methods in order to estimate Mean Opinion Scores (MOS) values. We have used information retrieved from the packet headers and side information provided as network parameters for model training. Random Forest based models have performed better in terms of Root Mean Square Error (RMSE) and Pearson correlation coefficient. The side information proved to be very effective in developing the model. We have found that, while the model performance might be improved by replacing the side information with more accurate bit stream level measurements, they are performing well in estimating perceived quality in audiovisual multimedia services.