Who Are We Talking About? Handling Person Names in Speech Translation

TL;DR

This paper identifies and addresses the poor handling of person names in speech translation systems, proposing multilingual models and joint transcription-translation training to significantly improve accuracy.

Contribution

It introduces a detailed analysis of person name errors and presents novel multilingual and joint training methods to enhance person name translation in speech translation systems.

Findings

47.8% relative improvement in person name accuracy

Analysis of nationality as a key factor in errors

Effective multilingual and joint training strategies

Abstract

Recent work has shown that systems for speech translation (ST) -- similarly to automatic speech recognition (ASR) -- poorly handle person names. This shortcoming does not only lead to errors that can seriously distort the meaning of the input, but also hinders the adoption of such systems in application scenarios (like computer-assisted interpreting) where the translation of named entities, like person names, is crucial. In this paper, we first analyse the outputs of ASR/ST systems to identify the reasons of failures in person name transcription/translation. Besides the frequency in the training data, we pinpoint the nationality of the referred person as a key factor. We then mitigate the problem by creating multilingual models, and further improve our ST systems by forcing them to jointly generate transcripts and translations, prioritising the former over the latter. Overall, our solutions result in a relative improvement in token-level person name accuracy by 47.8% on average for three language pairs (en->es,fr,it).