Finetuning End-to-End Models for Estonian Conversational Spoken Language Translation

TL;DR

This study explores finetuning end-to-end speech translation models for Estonian-English and Estonian-Russian, demonstrating that synthetic data significantly improves translation accuracy and can outperform traditional cascaded systems.

Contribution

The paper introduces a method for augmenting limited Estonian speech translation data with synthetic data and evaluates multiple models, showing improved performance over existing systems.

Findings

Synthetic data improves translation accuracy substantially.

SeamlessM4T matches or exceeds cascaded system performance.

Finetuning end-to-end models is effective for low-resource languages.

Abstract

This paper investigates the finetuning of end-to-end models for bidirectional Estonian-English and Estonian-Russian conversational speech-to-text translation. Due to the limited availability of speech translation data for Estonian, we created additional training data by web scraping and synthesizing data from speech recognition datasets using machine translation. We evaluated three publicly available end-to-end models: Whisper, OWSM 3.1, and SeamlessM4T. Our results indicate that fine-tuning with synthetic data enhances translation accuracy by a large margin, with SeamlessM4T matching or surpassing cascaded speech translation systems that use state-of-the-art speech recognition and machine translation models.