Lost in Interpreting: Speech Translation from Source or Interpreter?

TL;DR

This paper compares automatic speech translation systems that follow either the original speaker or the interpreter, analyzing their translation quality, delay, and information retention using a new European Parliament corpus.

Contribution

It introduces the Europarl Simultaneous Interpreting Corpus (ESIC) and evaluates the impact of following the speaker versus the interpreter on translation quality and latency.

Findings

Interpreter-based systems have higher delay but better information retention.

Speaker-based systems are faster but may lose more information.

Human evaluation shows differences in information loss between approaches.

Abstract

Interpreters facilitate multi-lingual meetings but the affordable set of languages is often smaller than what is needed. Automatic simultaneous speech translation can extend the set of provided languages. We investigate if such an automatic system should rather follow the original speaker, or an interpreter to achieve better translation quality at the cost of increased delay. To answer the question, we release Europarl Simultaneous Interpreting Corpus (ESIC), 10 hours of recordings and transcripts of European Parliament speeches in English, with simultaneous interpreting into Czech and German. We evaluate quality and latency of speaker-based and interpreter-based spoken translation systems from English to Czech. We study the differences in implicit simplification and summarization of the human interpreter compared to a machine translation system trained to shorten the output to some extent. Finally, we perform human evaluation to measure information loss of each of these approaches.