Disentangling ASR and MT Errors in Speech Translation

TL;DR

This paper proposes a method for automatically detecting and distinguishing errors originating from transcription and translation modules in speech translation systems, using a joint classifier and label extraction techniques.

Contribution

It introduces a novel approach to disentangle ASR and MT errors in speech translation, enabling more precise error analysis and quality assessment.

Findings

Effective joint classifier for 2-class and 3-class error detection

Successful label extraction methods for error source disentanglement

Qualitative analysis of error origins in speech translation output

Abstract

The main aim of this paper is to investigate automatic quality assessment for spoken language translation (SLT). More precisely, we investigate SLT errors that can be due to transcription (ASR) or to translation (MT) modules. This paper investigates automatic detection of SLT errors using a single classifier based on joint ASR and MT features. We evaluate both 2-class (good/bad) and 3-class (good/badASR/badMT ) labeling tasks. The 3-class problem necessitates to disentangle ASR and MT errors in the speech translation output and we propose two label extraction methods for this non trivial step. This enables - as a by-product - qualitative analysis on the SLT errors and their origin (are they due to transcription or to translation step?) on our large in-house corpus for French-to-English speech translation.