MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translation

TL;DR

This paper introduces MMTIT-Bench, a comprehensive multilingual and multi-scenario benchmark for text-image machine translation, and proposes CPR-Trans, a reasoning-oriented data paradigm that enhances translation accuracy and interpretability.

Contribution

The paper presents MMTIT-Bench, a new benchmark with diverse languages and scenarios, and introduces CPR-Trans, a novel cognition-perception-reasoning framework for improved TIMT.

Findings

CPR-Trans improves translation accuracy on 3B and 7B models.

Structured supervision enhances interpretability of VLLMs.

Benchmark enables rigorous evaluation across diverse languages and scenarios.

Abstract

End-to-end text-image machine translation (TIMT), which directly translates textual content in images across languages, is crucial for real-world multilingual scene understanding. Despite advances in vision-language large models (VLLMs), robustness across diverse visual scenes and low-resource languages remains underexplored due to limited evaluation resources. We present MMTIT-Bench, a human-verified multilingual and multi-scenario benchmark with 1,400 images spanning fourteen non-English and non-Chinese languages and diverse settings such as documents, scenes, and web images, enabling rigorous assessment of end-to-end TIMT. Beyond benchmarking, we study how reasoning-oriented data design improves translation. Although recent VLLMs have begun to incorporate long Chain-of-Thought (CoT) reasoning, effective thinking paradigms for TIMT are still immature: existing designs either cascade parsing and translation in a sequential manner or focus on language-only reasoning, overlooking the visual cognition central to VLLMs. We propose Cognition-Perception-Reasoning for Translation (CPR-Trans), a data paradigm that integrates scene cognition, text perception, and translation reasoning within a unified reasoning process. Using a VLLM-driven data generation pipeline, CPR-Trans provides structured, interpretable supervision that aligns perception with reasoning. Experiments on 3B and 7B models show consistent gains in accuracy and interpretability. We will release MMTIT-Bench to promote the multilingual and multi-scenario TIMT research upon acceptance.