Unify word-level and span-level tasks: NJUNLP's Participation for the WMT2023 Quality Estimation Shared Task

TL;DR

This paper presents NJUNLP's approach to the WMT2023 QE shared task, combining pseudo data generation, pre-training, and fine-tuning of models for improved quality estimation at sentence, word, and span levels.

Contribution

The paper introduces a unified framework that leverages pseudo data and joint learning for sentence, word, and span quality estimation tasks.

Findings

Achieved top results in English-German QE sub-tasks.

Effective use of pseudo MQM data for pre-training.

Proposed a simple method to convert word-level outputs to span-level results.

Abstract

We introduce the submissions of the NJUNLP team to the WMT 2023 Quality Estimation (QE) shared task. Our team submitted predictions for the English-German language pair on all two sub-tasks: (i) sentence- and word-level quality prediction; and (ii) fine-grained error span detection. This year, we further explore pseudo data methods for QE based on NJUQE framework (https://github.com/NJUNLP/njuqe). We generate pseudo MQM data using parallel data from the WMT translation task. We pre-train the XLMR large model on pseudo QE data, then fine-tune it on real QE data. At both stages, we jointly learn sentence-level scores and word-level tags. Empirically, we conduct experiments to find the key hyper-parameters that improve the performance. Technically, we propose a simple method that covert the word-level outputs to fine-grained error span results. Overall, our models achieved the best results in English-German for both word-level and fine-grained error span detection sub-tasks by a considerable margin.