Phone Features Improve Speech Translation

TL;DR

This paper compares cascade and end-to-end speech translation models across various resource conditions, introducing phone features to improve end-to-end models and close the performance gap with cascades, especially in low-resource scenarios.

Contribution

It demonstrates that incorporating phone features into end-to-end speech translation models significantly enhances their performance, surpassing previous methods and narrowing the gap with cascaded models.

Findings

Cascaded models remain stronger baselines across conditions.

Phone features improve end-to-end model performance.

Up to 9 BLEU improvement in low-resource settings.

Abstract

End-to-end models for speech translation (ST) more tightly couple speech recognition (ASR) and machine translation (MT) than a traditional cascade of separate ASR and MT models, with simpler model architectures and the potential for reduced error propagation. Their performance is often assumed to be superior, though in many conditions this is not yet the case. We compare cascaded and end-to-end models across high, medium, and low-resource conditions, and show that cascades remain stronger baselines. Further, we introduce two methods to incorporate phone features into ST models. We show that these features improve both architectures, closing the gap between end-to-end models and cascades, and outperforming previous academic work -- by up to 9 BLEU on our low-resource setting.