Bias-Aware Loss for Training Image and Speech Quality Prediction Models from Multiple Datasets

TL;DR

This paper introduces a bias-aware loss function for training image and speech quality prediction models that accounts for experiment-specific biases, improving model performance across multiple datasets.

Contribution

The paper proposes a novel bias-aware loss function that estimates and compensates for dataset biases during training of quality prediction models.

Findings

Improved accuracy of quality prediction models on synthetic and real datasets.

Effective estimation of dataset biases during training.

Enhanced model robustness across multiple datasets.

Abstract

The ground truth used for training image, video, or speech quality prediction models is based on the Mean Opinion Scores (MOS) obtained from subjective experiments. Usually, it is necessary to conduct multiple experiments, mostly with different test participants, to obtain enough data to train quality models based on machine learning. Each of these experiments is subject to an experiment-specific bias, where the rating of the same file may be substantially different in two experiments (e.g. depending on the overall quality distribution). These different ratings for the same distortion levels confuse neural networks during training and lead to lower performance. To overcome this problem, we propose a bias-aware loss function that estimates each dataset's biases during training with a linear function and considers it while optimising the network weights. We prove the efficiency of the proposed method by training and validating quality prediction models on synthetic and subjective image and speech quality datasets.