Prompting LLMs: Length Control for Isometric Machine Translation

TL;DR

This paper investigates how different prompting strategies and demonstration choices affect length control and translation quality in large language models for isometric machine translation across multiple language pairs.

Contribution

It demonstrates the importance of instruction phrasing and demonstration selection in controlling translation length and quality, achieving state-of-the-art results with multiple outputs.

Findings

Instruction phrasing aligned with demonstrations influences length control.

Extreme examples lead to shorter translations, but isometric demos often ignore length constraints.

Multiple outputs improve the length-quality tradeoff significantly.

Abstract

In this study, we explore the effectiveness of isometric machine translation across multiple language pairs (En$\to$De, En$\to$Fr, and En$\to$Es) under the conditions of the IWSLT Isometric Shared Task 2022. Using eight open-source large language models (LLMs) of varying sizes, we investigate how different prompting strategies, varying numbers of few-shot examples, and demonstration selection influence translation quality and length control. We discover that the phrasing of instructions, when aligned with the properties of the provided demonstrations, plays a crucial role in controlling the output length. Our experiments show that LLMs tend to produce shorter translations only when presented with extreme examples, while isometric demonstrations often lead to the models disregarding length constraints. While few-shot prompting generally enhances translation quality, further improvements are marginal across 5, 10, and 20-shot settings. Finally, considering multiple outputs allows to notably improve overall tradeoff between the length and quality, yielding state-of-the-art performance for some language pairs.