Cross-attention conformer for context modeling in speech enhancement for ASR

TL;DR

This paper proposes a cross-attention conformer architecture that leverages contextual information, such as preceding noise segments, to enhance speech features and improve robustness in automatic speech recognition systems.

Contribution

It introduces a novel cross-attention conformer model that effectively integrates sequential context information for speech enhancement.

Findings

Improved noise robustness in ASR using cross-attention conformer.

Effective merging of contextual noise information with input features.

Potential for deep contextual modeling in speech enhancement.

Abstract

This work introduces \emph{cross-attention conformer}, an attention-based architecture for context modeling in speech enhancement. Given that the context information can often be sequential, and of different length as the audio that is to be enhanced, we make use of cross-attention to summarize and merge contextual information with input features. Building upon the recently proposed conformer model that uses self attention layers as building blocks, the proposed cross-attention conformer can be used to build deep contextual models. As a concrete example, we show how noise context, i.e., short noise-only audio segment preceding an utterance, can be used to build a speech enhancement feature frontend using cross-attention conformer layers for improving noise robustness of automatic speech recognition.