Do End-to-End Speech Recognition Models Care About Context?

TL;DR

This paper compares CTC and AED end-to-end speech recognition models, revealing that AED is more context-sensitive but CTC can match its performance when enhanced, challenging assumptions about their reliance on context.

Contribution

The study demonstrates that CTC models can perform competitively without external language models and that their context sensitivity can be improved with self-attention, bridging the gap with AED models.

Findings

AED models are more context sensitive than CTC models.

Adding self-attention to CTC improves its context sensitivity.

CTC models perform well on WSJ and LibriSpeech without external language models.

Abstract

The two most common paradigms for end-to-end speech recognition are connectionist temporal classification (CTC) and attention-based encoder-decoder (AED) models. It has been argued that the latter is better suited for learning an implicit language model. We test this hypothesis by measuring temporal context sensitivity and evaluate how the models perform when we constrain the amount of contextual information in the audio input. We find that the AED model is indeed more context sensitive, but that the gap can be closed by adding self-attention to the CTC model. Furthermore, the two models perform similarly when contextual information is constrained. Finally, in contrast to previous research, our results show that the CTC model is highly competitive on WSJ and LibriSpeech without the help of an external language model.