A Comparative Study on End-to-end Speech to Text Translation

TL;DR

This paper compares various end-to-end speech-to-text translation architectures, explores auxiliary loss functions and pre-training methods, and demonstrates improved performance on multiple speech translation benchmarks.

Contribution

It provides a comprehensive overview of end-to-end speech translation models, investigates the impact of auxiliary CTC loss and pre-training, and achieves state-of-the-art results.

Findings

Pre-training boosts BLEU by up to 4% and TER by 5%.

Auxiliary CTC loss improves convergence.

Achieves new state-of-the-art results on IWSLT and LibriSpeech tasks.

Abstract

Recent advances in deep learning show that end-to-end speech to text translation model is a promising approach to direct the speech translation field. In this work, we provide an overview of different end-to-end architectures, as well as the usage of an auxiliary connectionist temporal classification (CTC) loss for better convergence. We also investigate on pre-training variants such as initializing different components of a model using pre-trained models, and their impact on the final performance, which gives boosts up to 4% in BLEU and 5% in TER. Our experiments are performed on 270h IWSLT TED-talks En->De, and 100h LibriSpeech Audiobooks En->Fr. We also show improvements over the current end-to-end state-of-the-art systems on both tasks.