MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation

TL;DR

This paper introduces MIMO-DBnet, an end-to-end neural network for speech separation that uses multi-channel input and multiple outputs to estimate directions and beamforming weights, improving spatial discrimination without extra cues.

Contribution

The novel MIMO-DBnet architecture enables direction-guided speech separation using only mixture signals, effectively handling phase wrapping issues and enhancing separation accuracy.

Findings

Achieves significant improvement over baseline systems.

Maintains high-frequency performance despite phase wrapping.

Provides effective spatial discrimination guidance.

Abstract

Recently, many deep learning based beamformers have been proposed for multi-channel speech separation. Nevertheless, most of them rely on extra cues known in advance, such as speaker feature, face image or directional information. In this paper, we propose an end-to-end beamforming network for direction guided speech separation given merely the mixture signal, namely MIMO-DBnet. Specifically, we design a multi-channel input and multiple outputs architecture to predict the direction-of-arrival based embeddings and beamforming weights for each source. The precisely estimated directional embedding provides quite effective spatial discrimination guidance for the neural beamformer to offset the effect of phase wrapping, thus allowing more accurate reconstruction of two sources' speech signals. Experiments show that our proposed MIMO-DBnet not only achieves a comprehensive decent improvement compared to baseline systems, but also maintain the performance on high frequency bands when phase wrapping occurs.