Speaker and Direction Inferred Dual-channel Speech Separation

TL;DR

This paper introduces SDNet, a novel speech separation model inspired by auditory attention, capable of handling unknown numbers of sources and improving separation accuracy using spatial features.

Contribution

The paper proposes SDNet, a new dual-channel speech separation network that infers speaker and direction information to enhance separation in dynamic, real-world scenarios.

Findings

Achieves SDR improvements of 25.31 dB, 17.26 dB, and 21.56 dB on standard benchmarks.

Effectively handles unknown number of sources and source selection.

Demonstrates superior performance over existing methods in fully-overlapped speech separation.

Abstract

Most speech separation methods, trying to separate all channel sources simultaneously, are still far from having enough general- ization capabilities for real scenarios where the number of input sounds is usually uncertain and even dynamic. In this work, we employ ideas from auditory attention with two ears and propose a speaker and direction inferred speech separation network (dubbed SDNet) to solve the cocktail party problem. Specifically, our SDNet first parses out the respective perceptual representations with their speaker and direction characteristics from the mixture of the scene in a sequential manner. Then, the perceptual representations are utilized to attend to each corresponding speech. Our model gener- ates more precise perceptual representations with the help of spatial features and successfully deals with the problem of the unknown number of sources and the selection of outputs. The experiments on standard fully-overlapped speech separation benchmarks, WSJ0- 2mix, WSJ0-3mix, and WSJ0-2&3mix, show the effectiveness, and our method achieves SDR improvements of 25.31 dB, 17.26 dB, and 21.56 dB under anechoic settings. Our codes will be released at https://github.com/aispeech-lab/SDNet.