Exploring Large Language Models for Translating Romanian Computational Problems into English

TL;DR

This paper investigates the effectiveness of large language models in translating Romanian computational problems into English, demonstrating that with proper prompts and supervision, they can reliably support multilingual problem-solving and educational applications.

Contribution

The study evaluates multiple LLMs for translating Romanian computational tasks, introduces an augmented dataset with English translations, and compares LLM performance against human translators.

Findings

LLMs can maintain or improve translation accuracy with well-structured prompts.

Augmented datasets enhance LLM training and evaluation for multilingual tasks.

Human oversight is crucial for ensuring translation quality in critical applications.

Abstract

Recent studies have suggested that large language models (LLMs) underperform on mathematical and computer science tasks when these problems are translated from Romanian into English, compared to their original Romanian format. Accurate translation is critical for applications ranging from automatic translations in programming competitions to the creation of high-quality educational materials, as well as minimizing errors or fraud in human translations. This study shows that robust large language models (LLMs) can maintain or even enhance their performance in translating less common languages when given well-structured prompts. Our findings suggest that LLMs, with appropriate supervision, can be reliably used for the automatic translation of IOI (International Olympiad in Informatics)-style tasks. We evaluate several translation methods across multiple LLMs, including OpenRoLLM, Llama 3.1 8B, Llama 3.2 3B and GPT-4o, assessing their translation accuracy and performance stability through repeated runs. Additionally, we augment the OJI (Romanian County-Level Informatics Olympiad) Romanian dataset with accurate English translations, enhancing its utility for future LLM training and evaluation. Through detailed syntactic and semantic analyses, we confirm that with human oversight, LLMs can serve as a viable solution for multilingual problem-solving. We also compare the translation quality of LLMs against human translators, as evaluated by a certified expert, underscoring the potential of LLMs in realworld scenarios.