Independence-based Joint Dereverberation and Separation with Neural Source Model

TL;DR

This paper introduces an end-to-end neural network approach for joint dereverberation and source separation that is permutation-invariant and effective across varying numbers of speakers and microphones, improving speech quality and recognition accuracy.

Contribution

It extends independent vector analysis with a neural source model trained end-to-end, enabling robust joint dereverberation and separation regardless of speaker count.

Findings

Achieves high speech quality and low WER in experiments.

Effective on synthetic and real recorded datasets without modifications.

Handles varying numbers of speakers and microphones.

Abstract

We propose an independence-based joint dereverberation and separation method with a neural source model. We introduce a neural network in the framework of time-decorrelation iterative source steering, which is an extension of independent vector analysis to joint dereverberation and separation. The network is trained in an end-to-end manner with a permutation invariant loss on the time-domain separation output signals. Our proposed method can be applied in any situation with at least as many microphones as sources, regardless of their number. In experiments, we demonstrate that our method results in high performance in terms of both speech quality metrics and word error rate (WER), even for mixtures with a different number of speakers than training. Furthermore, the model, trained on synthetic mixtures, without any modifications, greatly reduces the WER on the recorded dataset LibriCSS.