Adapting End-to-End Speech Recognition for Readable Subtitles

TL;DR

This paper explores adapting end-to-end speech recognition systems to produce readable subtitles by incorporating output compression and length constraints, improving subtitle readability with limited training data.

Contribution

It introduces methods for integrating compression and length modeling into end-to-end ASR, enabling readable subtitle generation with minimal additional data.

Findings

End-to-end models can be adapted for output compression with limited data.

Explicit length modeling improves subtitle readability and recognition performance.

Unsupervised post-editing can enhance transcription quality for subtitling.

Abstract

Automatic speech recognition (ASR) systems are primarily evaluated on transcription accuracy. However, in some use cases such as subtitling, verbatim transcription would reduce output readability given limited screen size and reading time. Therefore, this work focuses on ASR with output compression, a task challenging for supervised approaches due to the scarcity of training data. We first investigate a cascaded system, where an unsupervised compression model is used to post-edit the transcribed speech. We then compare several methods of end-to-end speech recognition under output length constraints. The experiments show that with limited data far less than needed for training a model from scratch, we can adapt a Transformer-based ASR model to incorporate both transcription and compression capabilities. Furthermore, the best performance in terms of WER and ROUGE scores is achieved by explicitly modeling the length constraints within the end-to-end ASR system.