Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition

TL;DR

This paper introduces a channel-aware GAN-based data simulation method to improve the robustness of speech recognition systems against unseen recording environments, significantly reducing error rates.

Contribution

It proposes a novel approach combining channel extraction and GANs to generate realistic target-domain speech for robust ASR training.

Findings

Achieved 20.02% relative CER reduction on HAT corpus.

Achieved 9.64% relative CER reduction on TAT corpus.

Demonstrated effectiveness of channel-aware data simulation in bridging domain gaps.

Abstract

While pre-trained automatic speech recognition (ASR) systems demonstrate impressive performance on matched domains, their performance often degrades when confronted with channel mismatch stemming from unseen recording environments and conditions. To mitigate this issue, we propose a novel channel-aware data simulation method for robust ASR training. Our method harnesses the synergistic power of channel-extractive techniques and generative adversarial networks (GANs). We first train a channel encoder capable of extracting embeddings from arbitrary audio. On top of this, channel embeddings are extracted using a minimal amount of target-domain data and used to guide a GAN-based speech synthesizer. This synthesizer generates speech that faithfully preserves the phonetic content of the input while mimicking the channel characteristics of the target domain. We evaluate our method on the challenging Hakka Across Taiwan (HAT) and Taiwanese Across Taiwan (TAT) corpora, achieving relative character error rate (CER) reductions of 20.02% and 9.64%, respectively, compared to the baselines. These results highlight the efficacy of our channel-aware data simulation method for bridging the gap between source- and target-domain acoustics.