Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation

TL;DR

This paper introduces a novel data simulation pipeline and training strategies to improve the generalization of speech separation models in diverse real-world acoustic environments, validated through extensive experiments.

Contribution

The paper presents AC-SIM, a new data simulation pipeline, and integrates multiple training objectives into PIT to enhance speech separation generalization across varied scenarios.

Findings

Significant improvement in generalization on real-world test sets

Enhanced separation quality across multiple architectures

Validated effectiveness through comprehensive objective and human listening tests

Abstract

Achieving robust speech separation for overlapping speakers in various acoustic environments with noise and reverberation remains an open challenge. Although existing datasets are available to train separators for specific scenarios, they do not effectively generalize across diverse real-world scenarios. In this paper, we present a novel data simulation pipeline that produces diverse training data from a range of acoustic environments and content, and propose new training paradigms to improve quality of a general speech separation model. Specifically, we first introduce AC-SIM, a data simulation pipeline that incorporates broad variations in both content and acoustics. Then we integrate multiple training objectives into the permutation invariant training (PIT) to enhance separation quality and generalization of the trained model. Finally, we conduct comprehensive objective and human listening experiments across separation architectures and benchmarks to validate our methods, demonstrating substantial improvement of generalization on both non-homologous and real-world test sets.