QAMRO: Quality-aware Adaptive Margin Ranking Optimization for Human-aligned Assessment of Audio Generation Systems

TL;DR

QAMRO is a novel framework that improves the evaluation of audio generation systems by aligning machine assessments more closely with human perception through adaptive ranking optimization.

Contribution

It introduces a new quality-aware adaptive margin ranking optimization method that integrates multiple regression perspectives for better human-aligned audio evaluation.

Findings

Outperforms baseline models in human evaluation alignment

Leverages pre-trained audio-text models like CLAP and Audiobox-Aesthetics

Achieves superior results on the AudioMOS Challenge 2025 dataset

Abstract

Evaluating audio generation systems, including text-to-music (TTM), text-to-speech (TTS), and text-to-audio (TTA), remains challenging due to the subjective and multi-dimensional nature of human perception. Existing methods treat mean opinion score (MOS) prediction as a regression problem, but standard regression losses overlook the relativity of perceptual judgments. To address this limitation, we introduce QAMRO, a novel Quality-aware Adaptive Margin Ranking Optimization framework that seamlessly integrates regression objectives from different perspectives, aiming to highlight perceptual differences and prioritize accurate ratings. Our framework leverages pre-trained audio-text models such as CLAP and Audiobox-Aesthetics, and is trained exclusively on the official AudioMOS Challenge 2025 dataset. It demonstrates superior alignment with human evaluations across all dimensions, significantly outperforming robust baseline models.