Towards Effective Disambiguation for Machine Translation with Large Language Models

TL;DR

This paper investigates the use of large language models for disambiguation in machine translation, proposing methods to enhance their ability to handle ambiguous sentences, and demonstrating competitive performance against state-of-the-art systems.

Contribution

It introduces two techniques— in-context learning and fine-tuning on curated datasets— to improve LLMs' disambiguation capabilities in machine translation.

Findings

Methods match or outperform DeepL and NLLB in four language directions.

Curated datasets and resources are publicly released.

Provides insights into adapting LLMs for better disambiguation in MT.

Abstract

Resolving semantic ambiguity has long been recognised as a central challenge in the field of Machine Translation. Recent work on benchmarking translation performance on ambiguous sentences has exposed the limitations of conventional Neural Machine Translation (NMT) systems, which fail to handle many such cases. Large language models (LLMs) have emerged as a promising alternative, demonstrating comparable performance to traditional NMT models while introducing new paradigms for controlling the target outputs. In this paper, we study the capabilities of LLMs to translate "ambiguous sentences" - i.e. those containing highly polysemous words and/or rare word senses. We also propose two ways to improve their disambiguation capabilities, through a) in-context learning and b) fine-tuning on carefully curated ambiguous datasets. Experiments show that our methods can match or outperform state-of-the-art systems such as DeepL and NLLB in four out of five language directions. Our research provides valuable insights into effectively adapting LLMs to become better disambiguators during Machine Translation. We release our curated disambiguation corpora and resources at https://data.statmt.org/ambiguous-europarl.