Beam-Guided TasNet: An Iterative Speech Separation Framework with Multi-Channel Output

TL;DR

This paper introduces Beam-Guided TasNet, an iterative multi-channel speech separation framework that enhances separation performance by integrating neural network-based separation with beamforming in a cyclic, mutually reinforcing manner.

Contribution

It proposes a novel cyclic framework allowing multi-channel input and output, enabling iterative refinement and improved performance over traditional Beam-TasNet and approaching oracle MVDR results.

Findings

Achieved SDR of 21.5 dB on spatialized WSJ0-2MIX

Exceeded baseline Beam-TasNet by 4.1 dB SDR

Narrowed performance gap with oracle MVDR to 2 dB

Abstract

Time-domain audio separation network (TasNet) has achieved remarkable performance in blind source separation (BSS). Classic multi-channel speech processing framework employs signal estimation and beamforming. For example, Beam-TasNet links multi-channel convolutional TasNet (MC-Conv-TasNet) with minimum variance distortionless response (MVDR) beamforming, which leverages the strong modeling ability of data-driven network and boosts the performance of beamforming with an accurate estimation of speech statistics. Such integration can be viewed as a directed acyclic graph by accepting multi-channel input and generating multi-source output. In this paper, we design a "multi-channel input, multi-channel multi-source output" (MIMMO) speech separation system entitled "Beam-Guided TasNet", where MC-Conv-TasNet and MVDR can interact and promote each other more compactly under a directed cyclic flow. Specifically, the first stage uses Beam-TasNet to generate estimated single-speaker signals, which favors the separation in the second stage. The proposed framework facilitates iterative signal refinement with the guide of beamforming and seeks to reach the upper bound of the MVDR-based methods. Experimental results on the spatialized WSJ0-2MIX demonstrate that the Beam-Guided TasNet has achieved an SDR of 21.5 dB, exceeding the baseline Beam-TasNet by 4.1 dB under the same model size and narrowing the gap with the oracle signal-based MVDR to 2 dB.