Dual-Path Style Learning for End-to-End Noise-Robust Speech Recognition

TL;DR

This paper introduces a dual-path style learning method for end-to-end noise-robust speech recognition, effectively recovering suppressed speech information and improving accuracy under noisy conditions.

Contribution

It proposes a novel dual-path style learning framework that incorporates clean speech features and style mapping to enhance noise robustness in ASR systems.

Findings

Achieves 10.6% and 8.6% relative WER reduction on RATS and CHiME-4 datasets.

Effectively recovers suppressed speech information from noisy inputs.

Improves noise robustness by minimizing output differences in dual-path ASR.

Abstract

Automatic speech recognition (ASR) systems degrade significantly under noisy conditions. Recently, speech enhancement (SE) is introduced as front-end to reduce noise for ASR, but it also suppresses some important speech information, i.e., over-suppression. To alleviate this, we propose a dual-path style learning approach for end-to-end noise-robust speech recognition (DPSL-ASR). Specifically, we first introduce clean speech feature along with the fused feature from IFF-Net as dual-path inputs to recover the suppressed information. Then, we propose style learning to map the fused feature close to clean feature, in order to learn latent speech information from the latter, i.e., clean "speech style". Furthermore, we also minimize the distance of final ASR outputs in two paths to improve noise-robustness. Experiments show that the proposed approach achieves relative word error rate (WER) reductions of 10.6% and 8.6% over the best IFF-Net baseline, on RATS and CHiME-4 datasets respectively.