"Don't Teach Minerva": Guiding LLMs Through Complex Syntax for Faithful Latin Translation with RAG

TL;DR

This paper presents a reproducible pipeline using open-source LLMs and retrieval-augmented generation to improve Latin translation, achieving results comparable to proprietary systems without fine-tuning.

Contribution

Introduces a draft-based refinement pipeline with RAG for Latin translation, enhancing open-source LLM performance to match top proprietary models.

Findings

Open-source RAG system matches GPT-5 performance

Robust translation on in-domain and out-of-domain Latin texts

Pipeline and datasets released for reproducibility

Abstract

Translating a morphology-rich, low-resource language like Latin poses significant challenges. This paper introduces a reproducible draft-based refinement pipeline that elevates open-source Large Language Models (LLMs) to a performance level statistically comparable to top-tier proprietary systems. Our method first uses a fine-tuned NLLB-1.3B model to generate a high-quality, structurally faithful draft. A zero-shot LLM (Llama-3.3 or Qwen3) then polishes this draft, a process that can be further enhanced by augmenting the context with retrieved out-context examples (RAG). We demonstrate the robustness of this approach on two distinct benchmarks: a standard in-domain test set (Rosenthal, 2023) and a new, challenging out-of-domain (OOD) set of 12th-century Latin letters (2025). Our central finding is that this open-source RAG system achieves performance statistically comparable to the GPT-5 baseline, without any task-specific LLM fine-tuning. We release the pipeline, the Chartres OOD set, and evaluation scripts and models to facilitate replicability and further research.