Self-critical Sequence Training for Automatic Speech Recognition

TL;DR

This paper introduces self-critical sequence training (SCST), a reinforcement learning approach for speech recognition that aligns training with evaluation metrics, reducing mismatch issues and improving WER performance.

Contribution

It proposes a novel RL-based training method that directly optimizes WER and eliminates teacher-forcing dependence in sequence-to-sequence ASR models.

Findings

SCST achieves 8.7% relative WER reduction on clean speech

SCST achieves 7.8% relative WER reduction on noisy speech

The method improves training-test alignment in ASR models

Abstract

Although automatic speech recognition (ASR) task has gained remarkable success by sequence-to-sequence models, there are two main mismatches between its training and testing that might lead to performance degradation: 1) The typically used cross-entropy criterion aims to maximize log-likelihood of the training data, while the performance is evaluated by word error rate (WER), not log-likelihood; 2) The teacher-forcing method leads to the dependence on ground truth during training, which means that model has never been exposed to its own prediction before testing. In this paper, we propose an optimization method called self-critical sequence training (SCST) to make the training procedure much closer to the testing phase. As a reinforcement learning (RL) based method, SCST utilizes a customized reward function to associate the training criterion and WER. Furthermore, it removes the reliance on teacher-forcing and harmonizes the model with respect to its inference procedure. We conducted experiments on both clean and noisy speech datasets, and the results show that the proposed SCST respectively achieves 8.7% and 7.8% relative improvements over the baseline in terms of WER.