MBNet: MOS Prediction for Synthesized Speech with Mean-Bias Network

TL;DR

This paper introduces MBNet, a novel MOS prediction model that utilizes individual judge scores by modeling mean and bias components, leading to improved correlation with human judgments in synthesized speech quality assessment.

Contribution

MBNet is the first model to explicitly incorporate individual judge bias scores alongside mean scores for MOS prediction, enhancing the utilization of training data.

Findings

MBNet outperforms MOSNet baseline in system-level SRCC.

MBNet achieves 2.9% and 6.7% SRCC improvements on VCC 2018 and 2016 datasets.

Model effectively captures individual judge biases.

Abstract

Mean opinion score (MOS) is a popular subjective metric to assess the quality of synthesized speech, and usually involves multiple human judges to evaluate each speech utterance. To reduce the labor cost in MOS test, multiple methods have been proposed to automatically predict MOS scores. To our knowledge, for a speech utterance, all previous works only used the average of multiple scores from different judges as the training target and discarded the score of each individual judge, which did not well exploit the precious MOS training data. In this paper, we propose MBNet, a MOS predictor with a mean subnet and a bias subnet to better utilize every judge score in MOS datasets, where the mean subnet is used to predict the mean score of each utterance similar to that in previous works, and the bias subnet to predict the bias score (the difference between the mean score and each individual judge score) and capture the personal preference of individual judges. Experiments show that compared with MOSNet baseline that only leverages mean score for training, MBNet improves the system-level spearmans rank correlation co-efficient (SRCC) by 2.9% on VCC 2018 dataset and 6.7% on VCC 2016 dataset.