Demystifying TasNet: A Dissecting Approach

TL;DR

This paper analyzes the components of TasNet by gradually replacing parts of a frequency domain separation system, revealing insights into its performance, advantages, and limitations in different acoustic environments.

Contribution

It introduces a dissecting approach to understand TasNet's gains, linking its loss function to MSE, and evaluates its generalization to reverberant conditions.

Findings

Intermediate variants achieve comparable SDR gains to TasNet.

The si-SDR loss function relates to a logarithmic MSE, explaining performance.

Reverberant conditions reduce the effectiveness of TasNet's gains.

Abstract

In recent years time domain speech separation has excelled over frequency domain separation in single channel scenarios and noise-free environments. In this paper we dissect the gains of the time-domain audio separation network (TasNet) approach by gradually replacing components of an utterance-level permutation invariant training (u-PIT) based separation system in the frequency domain until the TasNet system is reached, thus blending components of frequency domain approaches with those of time domain approaches. Some of the intermediate variants achieve comparable signal-to-distortion ratio (SDR) gains to TasNet, but retain the advantage of frequency domain processing: compatibility with classic signal processing tools such as frequency-domain beamforming and the human interpretability of the masks. Furthermore, we show that the scale invariant signal-to-distortion ratio (si-SDR) criterion used as loss function in TasNet is related to a logarithmic mean square error criterion and that it is this criterion which contributes most reliable to the performance advantage of TasNet. Finally, we critically assess which gains in a noise-free single channel environment generalize to more realistic reverberant conditions.