Towards Reliable Machine Translation: Scaling LLMs for Critical Error Detection and Safety

TL;DR

This paper investigates how instruction-tuned Large Language Models can be scaled and adapted to detect critical errors in machine translation, enhancing safety and trustworthiness in multilingual AI systems.

Contribution

It demonstrates that scaling and adaptation of LLMs significantly improve critical error detection in machine translation, surpassing traditional encoder-only models.

Findings

Model scaling improves error detection accuracy

Adaptation strategies like zero-shot and fine-tuning are effective

Enhanced error detection contributes to safer multilingual AI systems

Abstract

Machine Translation (MT) plays a pivotal role in cross-lingual information access, public policy communication, and equitable knowledge dissemination. However, critical meaning errors, such as factual distortions, intent reversals, or biased translations, can undermine the reliability, fairness, and safety of multilingual systems. In this work, we explore the capacity of instruction-tuned Large Language Models (LLMs) to detect such critical errors, evaluating models across a range of parameters using the publicly accessible data sets. Our findings show that model scaling and adaptation strategies (zero-shot, few-shot, fine-tuning) yield consistent improvements, outperforming encoder-only baselines like XLM-R and ModernBERT. We argue that improving critical error detection in MT contributes to safer, more trustworthy, and socially accountable information systems by reducing the risk of disinformation, miscommunication, and linguistic harm, especially in high-stakes or underrepresented contexts. This work positions error detection not merely as a technical challenge, but as a necessary safeguard in the pursuit of just and responsible multilingual AI. The code will be made available at GitHub.