A Universally-Deployable ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement, and Voice Separation

TL;DR

This paper introduces a robust, universally-deployable speech frontend model that integrates acoustic echo cancellation, speech enhancement, and voice separation, with novel architectural improvements and strategies for handling missing contextual signals.

Contribution

It presents a generalized, improved joint model architecture with Signal Dropout for robust performance across varying contextual signal availability.

Findings

25.0% relative WER reduction on background speech without speaker embedding

61.2% relative WER reduction on AEC without device playback

Model performs nearly as well as task-specific models when signals are missing

Abstract

Recent work has shown that it is possible to train a single model to perform joint acoustic echo cancellation (AEC), speech enhancement, and voice separation, thereby serving as a unified frontend for robust automatic speech recognition (ASR). The joint model uses contextual information, such as a reference of the playback audio, noise context, and speaker embedding. In this work, we propose a number of novel improvements to such a model. First, we improve the architecture of the Cross-Attention Conformer that is used to ingest noise context into the model. Second, we generalize the model to be able to handle varying lengths of noise context. Third, we propose Signal Dropout, a novel strategy that models missing contextual information. In the absence of one or more signals, the proposed model performs nearly as well as task-specific models trained without these signals; and when such signals are present, our system compares well against systems that require all context signals. Over the baseline, the final model retains a relative word error rate reduction of 25.0% on background speech when speaker embedding is absent, and 61.2% on AEC when device playback is absent.