Few-Shot and Pseudo-Label Guided Speech Quality Evaluation with Large Language Models

TL;DR

GatherMOS introduces a large language model-based framework for speech quality evaluation, effectively aggregating diverse signals and outperforming existing methods, especially with limited labeled data.

Contribution

The paper presents GatherMOS, a novel LLM-based framework that combines acoustic descriptors and pseudo-labels for improved speech quality assessment.

Findings

GatherMOS outperforms DNSMOS, VQScore, and other models on VoiceBank-DEMAND.

Zero-shot GatherMOS maintains stable performance across conditions.

Few-shot guidance significantly improves results with matching support samples.

Abstract

In this paper, we introduce GatherMOS, a novel framework that leverages large language models (LLM) as meta-evaluators to aggregate diverse signals into quality predictions. GatherMOS integrates lightweight acoustic descriptors with pseudo-labels from DNSMOS and VQScore, enabling the LLM to reason over heterogeneous inputs and infer perceptual mean opinion scores (MOS). We further explore both zero-shot and few-shot in-context learning setups, showing that zero-shot GatherMOS maintains stable performance across diverse conditions, while few-shot guidance yields large gains when support samples match the test conditions. Experiments on the VoiceBank-DEMAND dataset demonstrate that GatherMOS consistently outperforms DNSMOS, VQScore, naive score averaging, and even learning-based models such as CNN-BLSTM and MOS-SSL when trained under limited labeled-data conditions. These results highlight the potential of LLM-based aggregation as a practical strategy for non-intrusive speech quality evaluation.