CHILL at SemEval-2025 Task 2: You Can't Just Throw Entities and Hope -- Make Your LLM to Get Them Right

TL;DR

This paper presents a novel approach for entity-aware machine translation that combines retrieval augmented generation and self-refinement with LLMs, including a self-evaluation mechanism to enhance entity translation accuracy.

Contribution

It introduces a new system integrating RAG and iterative self-assessment for improved entity translation in machine translation tasks.

Findings

Enhanced entity translation accuracy demonstrated

Self-evaluation improves overall translation quality

Effective combination of RAG and self-refinement techniques

Abstract

In this paper, we describe our approach for the SemEval 2025 Task 2 on Entity-Aware Machine Translation (EA-MT). Our system aims to improve the accuracy of translating named entities by combining two key approaches: Retrieval Augmented Generation (RAG) and iterative self-refinement techniques using Large Language Models (LLMs). A distinctive feature of our system is its self-evaluation mechanism, where the LLM assesses its own translations based on two key criteria: the accuracy of entity translations and overall translation quality. We demonstrate how these methods work together and effectively improve entity handling while maintaining high-quality translations.