Multichannel Speech Separation with Narrow-band Conformer

TL;DR

This paper introduces NBC, a multichannel speech separation method using narrow-band Conformer that processes each frequency independently to effectively cluster spatial vectors and improve separation performance.

Contribution

The paper presents a novel narrow-band Conformer network that leverages frequency-independent processing for multichannel speech separation, outperforming existing methods.

Findings

Achieves superior speech separation accuracy compared to state-of-the-art methods.

Effectively exploits spatial vector clustering through the Conformer architecture.

Demonstrates large margin improvements in experimental evaluations.

Abstract

This work proposes a multichannel speech separation method with narrow-band Conformer (named NBC). The network is trained to learn to automatically exploit narrow-band speech separation information, such as spatial vector clustering of multiple speakers. Specifically, in the short-time Fourier transform (STFT) domain, the network processes each frequency independently, and is shared by all frequencies. For one frequency, the network inputs the STFT coefficients of multichannel mixture signals, and predicts the STFT coefficients of separated speech signals. Clustering of spatial vectors shares a similar principle with the self-attention mechanism in the sense of computing the similarity of vectors and then aggregating similar vectors. Therefore, Conformer would be especially suitable for the present problem. Experiments show that the proposed narrow-band Conformer achieves better speech separation performance than other state-of-the-art methods by a large margin.