End-to-End Speech Translation with Knowledge Distillation

TL;DR

This paper introduces a knowledge distillation method to enhance end-to-end speech translation models by leveraging a trained text translation teacher, resulting in significant BLEU score improvements on multiple language pairs.

Contribution

The paper presents a novel application of knowledge distillation from text translation models to improve end-to-end speech translation performance.

Findings

End-to-end speech translation is feasible for similar and dissimilar language pairs.

Knowledge distillation improves BLEU scores by over 3.5 points.

The approach reduces error propagation and model size.

Abstract

End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-end ST models have advantages of lower latency, smaller model size and less error propagation. However, the combination of speech recognition and text translation in one model is more difficult than each of these two tasks. In this paper, we propose a knowledge distillation approach to improve ST model by transferring the knowledge from text translation model. Specifically, we first train a text translation model, regarded as a teacher model, and then ST model is trained to learn output probabilities from teacher model through knowledge distillation. Experiments on English- French Augmented LibriSpeech and English-Chinese TED corpus show that end-to-end ST is possible to implement on both similar and dissimilar language pairs. In addition, with the instruction of teacher model, end-to-end ST model can gain significant improvements by over 3.5 BLEU points.