Bring the Noise: Introducing Noise Robustness to Pretrained Automatic Speech Recognition

TL;DR

This paper introduces Cleancoder, a noise-robust preprocessor for speech recognition that can be integrated with various encoder-decoder models to improve accuracy in noisy environments.

Contribution

It presents a novel, architecture-agnostic denoising preprocessor that enhances automatic speech recognition performance under noisy conditions.

Findings

Cleancoder effectively filters noise from speech signals.

Improves Word Error Rate (WER) in noisy environments.

Works with both pretrained and training-from-scratch ASR models.

Abstract

In recent research, in the domain of speech processing, large End-to-End (E2E) systems for Automatic Speech Recognition (ASR) have reported state-of-the-art performance on various benchmarks. These systems intrinsically learn how to handle and remove noise conditions from speech. Previous research has shown, that it is possible to extract the denoising capabilities of these models into a preprocessor network, which can be used as a frontend for downstream ASR models. However, the proposed methods were limited to specific fully convolutional architectures. In this work, we propose a novel method to extract the denoising capabilities, that can be applied to any encoder-decoder architecture. We propose the Cleancoder preprocessor architecture that extracts hidden activations from the Conformer ASR model and feeds them to a decoder to predict denoised spectrograms. We train our pre-processor on the Noisy Speech Database (NSD) to reconstruct denoised spectrograms from noisy inputs. Then, we evaluate our model as a frontend to a pretrained Conformer ASR model as well as a frontend to train smaller Conformer ASR models from scratch. We show that the Cleancoder is able to filter noise from speech and that it improves the total Word Error Rate (WER) of the downstream model in noisy conditions for both applications.