Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora

TL;DR

This paper presents a dual conditional cross-entropy filtering method for noisy parallel corpora, improving data quality and translation performance by penalizing divergent sentence pairs using inverse translation models trained on clean data.

Contribution

Introduces a novel dual cross-entropy filtering technique that enhances noisy parallel data quality, leading to improved translation models and top performance in shared tasks.

Findings

Higher BLEU scores with filtered data from Paracrawl

Achieved top ranking in WMT2018 shared task

Effective in selecting high-quality parallel sentence pairs

Abstract

In this work we introduce dual conditional cross-entropy filtering for noisy parallel data. For each sentence pair of the noisy parallel corpus we compute cross-entropy scores according to two inverse translation models trained on clean data. We penalize divergent cross-entropies and weigh the penalty by the cross-entropy average of both models. Sorting or thresholding according to these scores results in better subsets of parallel data. We achieve higher BLEU scores with models trained on parallel data filtered only from Paracrawl than with models trained on clean WMT data. We further evaluate our method in the context of the WMT2018 shared task on parallel corpus filtering and achieve the overall highest ranking scores of the shared task, scoring top in three out of four subtasks.