Understand, Solve and Translate: Bridging the Multilingual Mathematical Reasoning Gap

TL;DR

This paper introduces HRM8K, a bilingual math benchmark, and proposes UST, a method that uses English as an anchor to improve multilingual reasoning in LLMs, significantly reducing performance gaps.

Contribution

The paper presents HRM8K for evaluating multilingual math reasoning and UST, a novel approach that enhances reasoning accuracy by leveraging English as an anchor language.

Findings

UST improves Korean math reasoning performance by 10.91%.

Performance gap reduces from 11.6% to 0.7%.

Method generalizes across different Korean domains.

Abstract

Large language models (LLMs) demonstrate exceptional performance on complex reasoning tasks. However, despite their strong reasoning capabilities in high-resource languages (e.g., English and Chinese), a significant performance gap persists in other languages. To investigate this gap in Korean, we introduce HRM8K, a benchmark comprising 8,011 English-Korean parallel bilingual math problems. Through systematic analysis of model behaviors, we identify a key finding: these performance disparities stem primarily from difficulties in comprehending non-English inputs, rather than limitations in reasoning capabilities. Based on these findings, we propose UST (Understand, Solve, and Translate), a method that strategically uses English as an anchor for reasoning and solution generation. By fine-tuning the model on 130k synthetically generated data points, UST achieves a 10.91% improvement on the HRM8K benchmark and reduces the multilingual performance gap from 11.6% to 0.7%. Additionally, we show that improvements from UST generalize effectively to different Korean domains, demonstrating that capabilities acquired from machine-verifiable content can be generalized to other areas. We publicly release the benchmark, training dataset, and models.