End-to-End Automatic Speech Translation of Audiobooks

TL;DR

This paper explores end-to-end speech translation of audiobooks, demonstrating the feasibility of training compact models that translate source speech directly into target text, with different training scenarios based on transcription availability.

Contribution

It introduces a new corpus for audiobook speech translation and evaluates end-to-end models under varying transcription availability conditions.

Findings

End-to-end models can be trained efficiently for audiobook translation.

Using source transcriptions during training improves translation performance.

The corpus and baseline models are publicly released for future research.

Abstract

We investigate end-to-end speech-to-text translation on a corpus of audiobooks specifically augmented for this task. Previous works investigated the extreme case where source language transcription is not available during learning nor decoding, but we also study a midway case where source language transcription is available at training time only. In this case, a single model is trained to decode source speech into target text in a single pass. Experimental results show that it is possible to train compact and efficient end-to-end speech translation models in this setup. We also distribute the corpus and hope that our speech translation baseline on this corpus will be challenged in the future.