A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy References

TL;DR

This paper investigates how noisy references affect SI-SDR in speech separation, revealing limitations and proposing enhancement methods to improve separation quality, with mixed results on perceived noisiness and artifacts.

Contribution

It derives the impact of noisy references on SI-SDR and proposes reference enhancement techniques to mitigate noise effects in supervised speech separation.

Findings

Noise limits achievable SI-SDR.

Enhanced references reduce noise in outputs.

Processing references can introduce artifacts.

Abstract

This paper examines the implications of using the Scale-Invariant Signal-to-Distortion Ratio (SI-SDR) as both evaluation and training objective in supervised speech separation, when the training references contain noise, as is the case with the de facto benchmark WSJ0-2Mix. A derivation of the SI-SDR with noisy references reveals that noise limits the achievable SI-SDR, or leads to undesired noise in the separated outputs. To address this, a method is proposed to enhance references and augment the mixtures with WHAM!, aiming to train models that avoid learning noisy references. Two models trained on these enhanced datasets are evaluated with the non-intrusive NISQA.v2 metric. Results show reduced noise in separated speech but suggest that processing references may introduce artefacts, limiting overall quality gains. Negative correlation is found between SI-SDR and perceived noisiness across models on the WSJ0-2Mix and Libri2Mix test sets, underlining the conclusion from the derivation.