Interactive Feature Fusion for End-to-End Noise-Robust Speech Recognition

TL;DR

This paper introduces IFF-Net, an interactive feature fusion network that combines enhanced and original noisy speech features to improve noise-robust speech recognition, reducing word error rates effectively.

Contribution

The paper presents a novel IFF-Net architecture that effectively fuses features to mitigate over-suppression issues in speech enhancement for ASR.

Findings

Achieves 4.1% absolute WER reduction over baseline

Effectively complements missing information in over-suppressed features

Improves robustness of speech recognition in noisy environments

Abstract

Speech enhancement (SE) aims to suppress the additive noise from a noisy speech signal to improve the speech's perceptual quality and intelligibility. However, the over-suppression phenomenon in the enhanced speech might degrade the performance of downstream automatic speech recognition (ASR) task due to the missing latent information. To alleviate such problem, we propose an interactive feature fusion network (IFF-Net) for noise-robust speech recognition to learn complementary information from the enhanced feature and original noisy feature. Experimental results show that the proposed method achieves absolute word error rate (WER) reduction of 4.1% over the best baseline on RATS Channel-A corpus. Our further analysis indicates that the proposed IFF-Net can complement some missing information in the over-suppressed enhanced feature.