Single channel voice separation for unknown number of speakers under reverberant and noisy settings

TL;DR

This paper introduces a unified time-domain neural network for separating an unknown number of speakers in reverberant and noisy environments, with a classification branch estimating speaker count and multiple heads handling different speaker numbers.

Contribution

The novel model combines speaker classification and multiple separation heads in a unified network, effectively handling unknown speaker counts in challenging acoustic conditions.

Findings

Significantly outperforms baseline models in noisy and reverberant settings.

Introduces a new dataset with up to five speakers in noisy reverberant environments.

Demonstrates robustness across various acoustic scenarios.

Abstract

We present a unified network for voice separation of an unknown number of speakers. The proposed approach is composed of several separation heads optimized together with a speaker classification branch. The separation is carried out in the time domain, together with parameter sharing between all separation heads. The classification branch estimates the number of speakers while each head is specialized in separating a different number of speakers. We evaluate the proposed model under both clean and noisy reverberant set-tings. Results suggest that the proposed approach is superior to the baseline model by a significant margin. Additionally, we present a new noisy and reverberant dataset of up to five different speakers speaking simultaneously.