Uncertainty-Aware Mean Opinion Score Prediction

TL;DR

This paper introduces an uncertainty-aware approach to MOS prediction that models both aleatory and epistemic uncertainties, improving robustness and applicability in diverse real-world scenarios.

Contribution

It proposes a novel system that incorporates heteroscedastic regression and Monte Carlo dropout to model uncertainties in MOS prediction, addressing a key limitation of previous models.

Findings

Captures uncertainty effectively

Enables selective prediction and out-of-domain detection

Enhances practical utility in diverse environments

Abstract

Mean Opinion Score (MOS) prediction has made significant progress in specific domains. However, the unstable performance of MOS prediction models across diverse samples presents ongoing challenges in the practical application of these systems. In this paper, we point out that the absence of uncertainty modeling is a significant limitation hindering MOS prediction systems from applying to the real and open world. We analyze the sources of uncertainty in the MOS prediction task and propose to establish an uncertainty-aware MOS prediction system that models aleatory uncertainty and epistemic uncertainty by heteroscedastic regression and Monte Carlo dropout separately. The experimental results show that the system captures uncertainty well and is capable of performing selective prediction and out-of-domain detection. Such capabilities significantly enhance the practical utility of MOS systems in diverse real and open-world environments.