Fast-ULCNet: A fast and ultra low complexity network for single-channel speech enhancement

TL;DR

Fast-ULCNet is a modified speech enhancement neural network that reduces latency and complexity by replacing GRU layers with FastGRNNs, while maintaining state-of-the-art performance.

Contribution

This paper introduces Fast-ULCNet, a novel adaptation of ULCNet using FastGRNNs and a trainable filter to mitigate internal state drifting, achieving lower latency and model size.

Findings

Fast-ULCNet reduces model size by over 50%.

Latency decreases by 34% on average.

Performance remains comparable to the original ULCNet.

Abstract

Single-channel speech enhancement algorithms are often used in resource-constrained embedded devices, where low latency and low complexity designs gain more importance. In recent years, researchers have proposed a wide variety of novel solutions to this problem. In particular, a recent deep learning model named ULCNet is among the state-of-the-art approaches in this domain. This paper proposes an adaptation of ULCNet, by replacing its GRU layers with FastGRNNs, to reduce both computational latency and complexity. Furthermore, this paper shows empirical evidence on the performance decay of FastGRNNs in long audio signals during inference due to internal state drifting, and proposes a novel approach based on a trainable complementary filter to mitigate it. The resulting model, Fast-ULCNet, performs on par with the state-of-the-art original ULCNet architecture on a speech enhancement task, while reducing its model size by more than half and decreasing its latency by 34% on average.