Human and Automatic Evaluation of English-Hindi Machine Translation

TL;DR

This paper evaluates English-Hindi machine translation systems using both human and automatic metrics at various levels, comparing their effectiveness to guide system improvements.

Contribution

It presents a comparative analysis of human and automatic evaluation methods for machine translation quality assessment.

Findings

Automatic metrics show varying correlation with human judgments.

Evaluation at different levels (sentence, document, system) provides comprehensive insights.

Comparison highlights strengths and limitations of automatic evaluation methods.

Abstract

For the past 60 years, Research in machine translation is going on. For the development in this field, a lot of new techniques are being developed each day. As a result, we have witnessed development of many automatic machine translators. A manager of machine translation development project needs to know the performance increase/decrease, after changes have been done in his system. Due to this reason, a need for evaluation of machine translation systems was felt. In this article, we shall present the evaluation of some machine translators. This evaluation will be done by a human evaluator and by some automatic evaluation metrics, which will be done at sentence, document and system level. In the end we shall also discuss the comparison between the evaluations.