# OpenKiwi: An Open Source Framework for Quality Estimation

## TL;DR

OpenKiwi is an open source PyTorch framework designed for translation quality estimation, supporting training and testing of systems at word and sentence levels, achieving state-of-the-art results on benchmark datasets.

## Contribution

It introduces a comprehensive, open source framework that implements top-performing quality estimation systems from recent campaigns, enabling reproducibility and further research.

## Key findings

- Achieves state-of-the-art performance on word-level tasks.
- Attains near state-of-the-art results on sentence-level tasks.
- Provides a versatile tool for quality estimation research.

## Abstract

We introduce OpenKiwi, a PyTorch-based open source framework for translation quality estimation. OpenKiwi supports training and testing of word-level and sentence-level quality estimation systems, implementing the winning systems of the WMT 2015-18 quality estimation campaigns. We benchmark OpenKiwi on two datasets from WMT 2018 (English-German SMT and NMT), yielding state-of-the-art performance on the word-level tasks and near state-of-the-art in the sentence-level tasks.