DeepFilterNet: A Low Complexity Speech Enhancement Framework for Full-Band Audio based on Deep Filtering

TL;DR

DeepFilterNet is a low-complexity, two-stage speech enhancement framework that uses deep filtering and perceptually motivated spectral modeling to outperform complex mask methods and state-of-the-art models.

Contribution

The paper introduces DeepFilterNet, a novel two-stage deep filtering approach that enhances speech by modeling spectral envelope and periodic components with low complexity.

Findings

Outperforms complex mask-based methods across various frequency resolutions.

Demonstrates superior speech enhancement performance compared to existing state-of-the-art models.

Enforces network sparsity for low computational complexity.

Abstract

Complex-valued processing has brought deep learning-based speech enhancement and signal extraction to a new level. Typically, the process is based on a time-frequency (TF) mask which is applied to a noisy spectrogram, while complex masks (CM) are usually preferred over real-valued masks due to their ability to modify the phase. Recent work proposed to use a complex filter instead of a point-wise multiplication with a mask. This allows to incorporate information from previous and future time steps exploiting local correlations within each frequency band. In this work, we propose DeepFilterNet, a two stage speech enhancement framework utilizing deep filtering. First, we enhance the spectral envelope using ERB-scaled gains modeling the human frequency perception. The second stage employs deep filtering to enhance the periodic components of speech. Additionally to taking advantage of perceptual properties of speech, we enforce network sparsity via separable convolutions and extensive grouping in linear and recurrent layers to design a low complexity architecture. We further show that our two stage deep filtering approach outperforms complex masks over a variety of frequency resolutions and latencies and demonstrate convincing performance compared to other state-of-the-art models.