Neural Speech Separation Using Spatially Distributed Microphones

TL;DR

This paper introduces a neural network architecture for speech separation using spatially distributed microphones without prior knowledge of their number or arrangement, combining self-attention and BLSTM layers.

Contribution

A novel neural network design that handles unknown microphone configurations by interleaving self-attention and temporal layers for improved speech separation.

Findings

Significantly outperforms baseline multi-channel systems in speech recognition tasks.

Effectively exploits spatial and temporal information across varying microphone setups.

Demonstrates robustness to unknown microphone arrangements.

Abstract

This paper proposes a neural network based speech separation method using spatially distributed microphones. Unlike with traditional microphone array settings, neither the number of microphones nor their spatial arrangement is known in advance, which hinders the use of conventional multi-channel speech separation neural networks based on fixed size input. To overcome this, a novel network architecture is proposed that interleaves inter-channel processing layers and temporal processing layers. The inter-channel processing layers apply a self-attention mechanism along the channel dimension to exploit the information obtained with a varying number of microphones. The temporal processing layers are based on a bidirectional long short term memory (BLSTM) model and applied to each channel independently. The proposed network leverages information across time and space by stacking these two kinds of layers alternately. Our network estimates time-frequency (TF) masks for each speaker, which are then used to generate enhanced speech signals either with TF masking or beamforming. Speech recognition experimental results show that the proposed method significantly outperforms baseline multi-channel speech separation systems.