Learning to assess subjective impressions from speech

TL;DR

This paper introduces a neural network framework to assess subjective impressions in speech, focusing on personalized voice descriptors, and demonstrates that comparison-based training improves performance with limited data.

Contribution

The work proposes a novel approach for subjective speech impression assessment using personalized descriptors and comparison data, outperforming traditional absolute rating methods.

Findings

CCR training outperforms ACR training in assessment accuracy

Assessment models can be effectively trained with limited data

Personalized SVDs improve subjective impression evaluation

Abstract

We tackle a new task of training neural network models that can assess subjective impressions conveyed through speech and assign scores accordingly, inspired by the work on automatic speech quality assessment (SQA). Speech impressions are often described using phrases like `cute voice.' We define such phrases as subjective voice descriptors (SVDs). Focusing on the difference in usage scenarios between the proposed task and automatic SQA, we design a framework capable of accommodating SVDs personalized to each individual, such as `my favorite voice.' In this work, we compiled a dataset containing speech labels derived from both abosolute category ratings (ACR) and comparison category ratings (CCR). As an evaluation metric for assessment performance, we introduce ppref, the accuracy of the predicted score ordering of two samples on CCR test samples. Alongside the conventional model and learning methods based on ACR data, we also investigated RankNet learning using CCR data. We experimentally find that the ppref is moderate even with very limited training data. We also discover the CCR training is superior to the ACR training. These results support the idea that assessment models based on personalized SVDs, which typically must be trained on limited data, can be effectively learned from CCR data.