Perceptual Evaluation of 360 Audiovisual Quality and Machine Learning Predictions

TL;DR

This study evaluates the prediction of perceived 360 audiovisual quality using objective metrics and machine learning, demonstrating that SVM models with VMAF and AMBIQUAL metrics achieve high accuracy.

Contribution

It introduces a machine learning framework combining multiple quality metrics to predict subjective audiovisual quality in 360 content, with comprehensive evaluation methods.

Findings

SVM outperforms other models in prediction accuracy.

VMAF and AMBIQUAL combination yields the best results.

k-Fold validation provides higher performance metrics.

Abstract

In an earlier study, we gathered perceptual evaluations of the audio, video, and audiovisual quality for 360 audiovisual content. This paper investigates perceived audiovisual quality prediction based on objective quality metrics and subjective scores of 360 video and spatial audio content. Thirteen objective video quality metrics and three objective audio quality metrics were evaluated for five stimuli for each coding parameter. Four regression-based machine learning models were trained and tested here, i.e., multiple linear regression, decision tree, random forest, and support vector machine. Each model was constructed using a combination of audio and video quality metrics and two cross-validation methods (k-Fold and Leave-One-Out) were investigated and produced 312 predictive models. The results indicate that the model based on the evaluation of VMAF and AMBIQUAL is better than other combinations of audio-video quality metric. In this study, support vector machine provides higher performance using k-Fold (PCC = 0.909, SROCC = 0.914, and RMSE = 0.416). These results can provide insights for the design of multimedia quality metrics and the development of predictive models for audiovisual omnidirectional media.