Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

TL;DR

This paper introduces an unsupervised deep learning approach for multi-channel speech source separation that uses a probabilistic loss based on Kullback-Leibler Divergence, enabling effective training without clean signals.

Contribution

It proposes a novel unsupervised training method using a probabilistic loss function with KLD, incorporating a statistical SCM model for robustness against reverberation and noise.

Findings

Effective training with small datasets (1K utterances)

Robust separation in reverberant environments

Probabilistic training avoids overfitting to separation errors

Abstract

In this paper, we propose a multi-channel speech source separation with a deep neural network (DNN) which is trained under the condition that no clean signal is available. As an alternative to a clean signal, the proposed method adopts an estimated speech signal by an unsupervised speech source separation with a statistical model. As a statistical model of microphone input signal, we adopts a time-varying spatial covariance matrix (SCM) model which includes reverberation and background noise submodels so as to achieve robustness against reverberation and background noise. The DNN infers intermediate variables which are needed for constructing the time-varying SCM. Speech source separation is performed in a probabilistic manner so as to avoid overfitting to separation error. Since there are multiple intermediate variables, a loss function which evaluates a single intermediate variable is not applicable. Instead, the proposed method adopts a loss function which evaluates the output probabilistic signal directly based on Kullback-Leibler Divergence (KLD). Gradient of the loss function can be back-propagated into the DNN through all the intermediate variables. Experimental results under reverberant conditions show that the proposed method can train the DNN efficiently even when the number of training utterances is small, i.e., 1K.