TL;DR
LITRANSPROQA is a novel LLM-based, reference-free evaluation metric for literary translation that incorporates professional insights to better assess artistic and cultural quality, outperforming existing metrics.
Contribution
The paper introduces LITRANSPROQA, a new literary translation evaluation framework that integrates human expertise and leverages large language models for improved assessment accuracy.
Findings
LITRANSPROQA outperforms current metrics in correlation and adequacy.
Incorporating professional translator insights enhances evaluation performance.
The framework is effective across open-source LLMs like LLaMA and Qwen.
Abstract
The impact of Large Language Models (LLMs) has extended into literary domains. However, existing evaluation metrics for literature prioritize mechanical accuracy over artistic expression and tend to overrate machine translation as being superior to human translation from experienced professionals. In the long run, this bias could result in an irreversible decline in translation quality and cultural authenticity. In response to the urgent need for a specialized literary evaluation metric, we introduce LITRANSPROQA, a novel, reference-free, LLM-based question-answering framework designed for literary translation evaluation. LITRANSPROQA integrates humans in the loop to incorporate insights from professional literary translators and researchers, focusing on critical elements in literary quality assessment such as literary devices, cultural understanding, and authorial voice. Our extensive…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
