On TasNet for Low-Latency Single-Speaker Speech Enhancement

TL;DR

This paper demonstrates that TasNet, a time-domain neural network architecture, effectively enhances single-speaker speech, especially for modulated noise, outperforming existing systems and revealing insights into its internal representations and limitations.

Contribution

The study extends TasNet's application to speech enhancement, showing its superiority over state-of-the-art methods and analyzing its internal representations and performance limitations.

Findings

TasNet improves speech enhancement, especially for modulated noise.

TasNet learns an inner-domain representation with separable target and noise signals.

Performance drops with large frame hops, likely due to aliasing.

Abstract

In recent years, speech processing algorithms have seen tremendous progress primarily due to the deep learning renaissance. This is especially true for speech separation where the time-domain audio separation network (TasNet) has led to significant improvements. However, for the related task of single-speaker speech enhancement, which is of obvious importance, it is yet unknown, if the TasNet architecture is equally successful. In this paper, we show that TasNet improves state-of-the-art also for speech enhancement, and that the largest gains are achieved for modulated noise sources such as speech. Furthermore, we show that TasNet learns an efficient inner-domain representation, where target and noise signal components are highly separable. This is especially true for noise in terms of interfering speech signals, which might explain why TasNet performs so well on the separation task. Additionally, we show that TasNet performs poorly for large frame hops and conjecture that aliasing might be the main cause of this performance drop. Finally, we show that TasNet consistently outperforms a state-of-the-art single-speaker speech enhancement system.