A new hybrid metric for verifying parallel corpora of Arabic-English

TL;DR

This paper introduces a hybrid metric combining sentence length and compression code length to improve the verification of Arabic-English parallel corpora, effectively filtering noise and enhancing translation quality assessment.

Contribution

It presents a novel combined metric that outperforms individual techniques in verifying the quality of Arabic-English sentence pairs.

Findings

Improved accuracy in identifying satisfactory sentence pairs

Effective noise filtering and reduction of mis-translations

Enhanced overall quality of parallel corpora

Abstract

This paper discusses a new metric that has been applied to verify the quality in translation between sentence pairs in parallel corpora of Arabic-English. This metric combines two techniques, one based on sentence length and the other based on compression code length. Experiments on sample test parallel Arabic-English corpora indicate the combination of these two techniques improves accuracy of the identification of satisfactory and unsatisfactory sentence pairs compared to sentence length and compression code length alone. The new method proposed in this research is effective at filtering noise and reducing mis-translations resulting in greatly improved quality.