Enhanced Bilingual Evaluation Understudy

TL;DR

This paper enhances the BLEU evaluation metric for machine translation by making it more adaptable and better aligned with human judgment, considering linguistic variations like synonyms and word order.

Contribution

It introduces a modified BLEU metric that accounts for linguistic variations, improving correlation with human evaluations in machine translation assessment.

Findings

Improved correlation with human judgments

Enhanced robustness to linguistic variations

Better alignment with human evaluation methods

Abstract

Our research extends the Bilingual Evaluation Understudy (BLEU) evaluation technique for statistical machine translation to make it more adjustable and robust. We intend to adapt it to resemble human evaluation more. We perform experiments to evaluate the performance of our technique against the primary existing evaluation methods. We describe and show the improvements it makes over existing methods as well as correlation to them. When human translators translate a text, they often use synonyms, different word orders or style, and other similar variations. We propose an SMT evaluation technique that enhances the BLEU metric to consider variations such as those.