SAN: a robust end-to-end ASR model architecture

TL;DR

This paper introduces SAN, a novel Siamese Adversarial Network architecture for robust end-to-end automatic speech recognition, significantly improving accuracy on fuzzy audio inputs and achieving state-of-the-art results.

Contribution

The paper presents a new SAN architecture that leverages adversarial learning to enhance acoustic feature extraction in ASR, especially for fuzzy audio, outperforming existing models.

Findings

Achieved a 4.37 CER on AISHELL-1 without language model

Reduced CER by around 5% relative on AISHELL-1

Demonstrated the model's effectiveness on phoneme recognition

Abstract

In this paper, we propose a novel Siamese Adversarial Network (SAN) architecture for automatic speech recognition, which aims at solving the difficulty of fuzzy audio recognition. Specifically, SAN constructs two sub-networks to differentiate the audio feature input and then introduces a loss to unify the output distribution of these sub-networks. Adversarial learning enables the network to capture more essential acoustic features and helps the models achieve better performance when encountering fuzzy audio input. We conduct numerical experiments with the SAN model on several datasets for the automatic speech recognition task. All experimental results show that the siamese adversarial nets significantly reduce the character error rate (CER). Specifically, we achieve a new state of art 4.37 CER without language model on the AISHELL-1 dataset, which leads to around 5% relative CER reduction. To reveal the generality of the siamese adversarial net, we also conduct experiments on the phoneme recognition task, which also shows the superiority of the siamese adversarial network.