Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Model

TL;DR

This paper presents a Transformer-based post-processing model for automatic speech recognition that significantly improves word error rates, especially in noisy conditions, by leveraging data augmentation and pre-trained weights.

Contribution

Introduces a Transformer sequence-to-sequence model for ASR correction that outperforms traditional methods and approaches neural language model re-scoring performance.

Findings

Significant WER reduction on LibriSpeech benchmark.

Effective use of data augmentation and pre-trained weights.

Outperforms baseline with 6-gram re-scoring.

Abstract

In this work, we introduce a simple yet efficient post-processing model for automatic speech recognition (ASR). Our model has Transformer-based encoder-decoder architecture which "translates" ASR model output into grammatically and semantically correct text. We investigate different strategies for regularizing and optimizing the model and show that extensive data augmentation and the initialization with pre-trained weights are required to achieve good performance. On the LibriSpeech benchmark, our method demonstrates significant improvement in word error rate over the baseline acoustic model with greedy decoding, especially on much noisier dev-other and test-other portions of the evaluation dataset. Our model also outperforms baseline with 6-gram language model re-scoring and approaches the performance of re-scoring with Transformer-XL neural language model.