The VoiceMOS Challenge 2022
Wen-Chin Huang, Erica Cooper, Yu Tsao, Hsin-Min Wang, Tomoki Toda,, Junichi Yamagishi

TL;DR
The VoiceMOS Challenge 2022 aimed to advance automatic prediction of human-rated speech quality, involving diverse approaches and revealing the effectiveness of fine-tuning self-supervised models while highlighting challenges with unseen data.
Contribution
This paper introduces the first VoiceMOS Challenge, providing a benchmark for MOS prediction and analyzing the effectiveness of different approaches on diverse speech synthesis data.
Findings
Fine-tuning self-supervised speech models improves MOS prediction accuracy.
Predicting MOS for unseen speakers and systems remains challenging.
The challenge attracted 22 teams from academia and industry.
Abstract
We present the first edition of the VoiceMOS Challenge, a scientific event that aims to promote the study of automatic prediction of the mean opinion score (MOS) of synthetic speech. This challenge drew 22 participating teams from academia and industry who tried a variety of approaches to tackle the problem of predicting human ratings of synthesized speech. The listening test data for the main track of the challenge consisted of samples from 187 different text-to-speech and voice conversion systems spanning over a decade of research, and the out-of-domain track consisted of data from more recent systems rated in a separate listening test. Results of the challenge show the effectiveness of fine-tuning self-supervised speech models for the MOS prediction task, as well as the difficulty of predicting MOS ratings for unseen speakers and listeners, and for unseen systems in the out-of-domain…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
