Blind Speech Separation and Dereverberation using Neural Beamforming

TL;DR

This paper introduces a neural network-based system that performs simultaneous speech separation, dereverberation, and speaker identification, utilizing neural beamforming and complex-valued models for improved audio processing in meeting scenarios.

Contribution

The paper presents a novel integrated neural network architecture for blind speech separation, dereverberation, and speaker identification, including frequency and time-domain models and a block-online processing mode.

Findings

Achieves improved SI-SDR scores in speech separation.

Reduces WER in speech recognition tasks.

Enhances EER in speaker verification.

Abstract

In this paper, we present the Blind Speech Separation and Dereverberation (BSSD) network, which performs simultaneous speaker separation, dereverberation and speaker identification in a single neural network. Speaker separation is guided by a set of predefined spatial cues. Dereverberation is performed by using neural beamforming, and speaker identification is aided by embedding vectors and triplet mining. We introduce a frequency-domain model which uses complex-valued neural networks, and a time-domain variant which performs beamforming in latent space. Further, we propose a block-online mode to process longer audio recordings, as they occur in meeting scenarios. We evaluate our system in terms of Scale Independent Signal to Distortion Ratio (SI-SDR), Word Error Rate (WER) and Equal Error Rate (EER).