Improving Translation Quality by Selecting Better Data for LLM Fine-Tuning: A Comparative Analysis

TL;DR

This paper evaluates how different data selection methods affect machine translation fine-tuning for open LLMs, highlighting the importance of semantic data quality for improving translation performance.

Contribution

It provides a comparative analysis of five data selectors, demonstrating the superiority of semantic selectors over lexical and geometry-based heuristics in translation quality.

Findings

Semantic selectors outperform other heuristics.

Small differences in selected data significantly impact performance.

Data quality critically influences fine-tuning outcomes.

Abstract

We investigated the impact of data selection on machine translation fine-tuning for open LLMs. Using Japanese-English corpora, we compare five selectors: TF-IDF, COMET Kiwi, QuRate, FD-Score, and random selection, under controlled training conditions. We observed that semantic selectors consistently outperform lexical and geometry-based heuristics, and that even when the selected data differ by less than 3%, the impact on model performance is substantial, underscoring the sensitivity of fine-tuning to data quality.