Improving Speech Enhancement with Multi-Metric Supervision from Learned Quality Assessment

TL;DR

This paper introduces a speech enhancement training method that uses a learned speech quality assessment model predicting multiple metrics, leading to improved perceptual quality and better generalization compared to traditional objectives.

Contribution

The work proposes a novel SQA-guided training framework for speech enhancement that leverages multi-metric supervision, addressing limitations of conventional objectives and enabling training on real-world data.

Findings

SQA-guided training improves speech quality across multiple metrics.

The approach generalizes better to real-world data.

It outperforms traditional training objectives like SI-SNR.

Abstract

Speech quality assessment (SQA) aims to predict the perceived quality of speech signals under a wide range of distortions. It is inherently connected to speech enhancement (SE), which seeks to improve speech quality by removing unwanted signal components. While SQA models are widely used to evaluate SE performance, their potential to guide SE training remains underexplored. In this work, we investigate a training framework that leverages a SQA model, trained to predict multiple evaluation metrics from a public SE leaderboard, as a supervisory signal for SE. This approach addresses a key limitation of conventional SE objectives, such as SI-SNR, which often fail to align with perceptual quality and generalize poorly across evaluation metrics. Moreover, it enables training on real-world data where clean references are unavailable. Experiments on both simulated and real-world test sets show that SQA-guided training consistently improves performance across a range of quality metrics. Code and checkpoints are available at https://github.com/urgent-challenge/urgent2026_challenge_track2