Improving Self-Supervised Learning-based MOS Prediction Networks

TL;DR

This paper enhances self-supervised learning models for predicting Mean Opinion Scores (MOS) in speech systems, aiming to reduce reliance on costly human evaluations through various training and data processing improvements.

Contribution

It introduces specific data, training, and post-training techniques to improve MOS prediction accuracy of a self-supervised model based on wav2vec 2.0.

Findings

Improved prediction accuracy on Voice MOS challenge dataset

Effective use of transfer learning and data preprocessing

Enhanced model robustness with dropout accumulation and quantization

Abstract

MOS (Mean Opinion Score) is a subjective method used for the evaluation of a system's quality. Telecommunications (for voice and video), and speech synthesis systems (for generated speech) are a few of the many applications of the method. While MOS tests are widely accepted, they are time-consuming and costly since human input is required. In addition, since the systems and subjects of the tests differ, the results are not really comparable. On the other hand, a large number of previous tests allow us to train machine learning models that are capable of predicting MOS value. By automatically predicting MOS values, both the aforementioned issues can be resolved. The present work introduces data-, training- and post-training specific improvements to a previous self-supervised learning-based MOS prediction model. We used a wav2vec 2.0 model pre-trained on LibriSpeech, extended with LSTM and non-linear dense layers. We introduced transfer learning, target data preprocessing a two- and three-phase training method with different batch formulations, dropout accumulation (for larger batch sizes) and quantization of the predictions. The methods are evaluated using the shared synthetic speech dataset of the first Voice MOS challenge.