MAATS: A Multi-Agent Automated Translation System Based on MQM Evaluation

TL;DR

MAATS is a multi-agent translation system that uses MQM error categories for detailed error detection and iterative refinement, significantly improving translation quality across various languages and models.

Contribution

It introduces a multi-agent framework leveraging MQM for fine-grained error analysis and translation refinement, advancing beyond single-agent approaches.

Findings

Outperforms zero-shot and single-agent baselines in automatic and human evaluations.

Excels in semantic accuracy and handling linguistically distant language pairs.

Provides detailed error diagnosis and context-aware translation improvements.

Abstract

We present MAATS, a Multi Agent Automated Translation System that leverages the Multidimensional Quality Metrics (MQM) framework as a fine-grained signal for error detection and refinement. MAATS employs multiple specialized AI agents, each focused on a distinct MQM category (e.g., Accuracy, Fluency, Style, Terminology), followed by a synthesis agent that integrates the annotations to iteratively refine translations. This design contrasts with conventional single-agent methods that rely on self-correction. Evaluated across diverse language pairs and Large Language Models (LLMs), MAATS outperforms zero-shot and single-agent baselines with statistically significant gains in both automatic metrics and human assessments. It excels particularly in semantic accuracy, locale adaptation, and linguistically distant language pairs. Qualitative analysis highlights its strengths in multi-layered error diagnosis, omission detection across perspectives, and context-aware refinement. By aligning modular agent roles with interpretable MQM dimensions, MAATS narrows the gap between black-box LLMs and human translation workflows, shifting focus from surface fluency to deeper semantic and contextual fidelity.