PHRASED: Phrase Dictionary Biasing for Speech Translation

TL;DR

This paper introduces a phrase dictionary biasing technique to improve speech translation accuracy, especially for rare phrases, by leveraging phrase pairs and external phrase information across different models.

Contribution

It proposes a novel phrase dictionary biasing method applicable to various speech translation models, enhancing translation accuracy and phrase recall significantly.

Findings

21% relative improvement over phrase list biasing in streaming speech translation

85% relative improvement in phrase recall for multimodal large language models

Effective leveraging of external phrase information in speech translation

Abstract

Phrases are essential to understand the core concepts in conversations. However, due to their rare occurrence in training data, correct translation of phrases is challenging in speech translation tasks. In this paper, we propose a phrase dictionary biasing method to leverage pairs of phrases mapping from the source language to the target language. We apply the phrase dictionary biasing method to two types of widely adopted models, a transducer-based streaming speech translation model and a multimodal large language model. Experimental results show that the phrase dictionary biasing method outperforms phrase list biasing by 21% relatively for the streaming speech translation model. In addition, phrase dictionary biasing enables multimodal large language models to use external phrase information, achieving 85% relative improvement in phrase recall.