Speech Denoising by Accumulating Per-Frequency Modeling Fluctuations

TL;DR

This paper introduces an unsupervised audio denoising method that combines time and time-frequency domain processing, leveraging neural network fitting scores to effectively separate clean speech from noise in a single clip.

Contribution

The novel approach integrates time and frequency domain analysis with neural network fitting scores, enabling unsupervised denoising tailored to individual audio clips.

Findings

Outperforms existing denoising methods in experiments

Effective in unsupervised, clip-specific denoising scenarios

Code and samples publicly available for reproducibility

Abstract

We present a method for audio denoising that combines processing done in both the time domain and the time-frequency domain. Given a noisy audio clip, the method trains a deep neural network to fit this signal. Since the fitting is only partly successful and is able to better capture the underlying clean signal than the noise, the output of the network helps to disentangle the clean audio from the rest of the signal. This is done by accumulating a fitting score per time-frequency bin and applying the time-frequency domain filtering based on the obtained scores. The method is completely unsupervised and only trains on the specific audio clip that is being denoised. Our experiments demonstrate favorable performance in comparison to the literature methods. Our code and samples are available at github.com/mosheman5/DNP and as supplementary. Index Terms: Audio denoising; Unsupervised learning