GECToR -- Grammatical Error Correction: Tag, Not Rewrite

TL;DR

This paper introduces GECToR, a fast and effective grammatical error correction system using sequence tagging with Transformers, achieving state-of-the-art results with significantly improved inference speed.

Contribution

The paper presents a novel sequence tagging approach for GEC using Transformers, with custom token transformations and a two-stage fine-tuning process, outperforming previous methods.

Findings

Achieves F0.5 scores of 65.3/66.5 on CoNLL-2014 and 72.4/73.6 on BEA-2019.

Inference speed is up to 10 times faster than seq2seq GEC systems.

Code and models are publicly available.

Abstract

In this paper, we present a simple and efficient GEC sequence tagger using a Transformer encoder. Our system is pre-trained on synthetic data and then fine-tuned in two stages: first on errorful corpora, and second on a combination of errorful and error-free parallel corpora. We design custom token-level transformations to map input tokens to target corrections. Our best single-model/ensemble GEC tagger achieves an $F_{0.5}$ of 65.3/66.5 on CoNLL-2014 (test) and $F_{0.5}$ of 72.4/73.6 on BEA-2019 (test). Its inference speed is up to 10 times as fast as a Transformer-based seq2seq GEC system. The code and trained models are publicly available.