TL;DR
This paper introduces GECToR, a fast and effective grammatical error correction system using sequence tagging with Transformers, achieving state-of-the-art results with significantly improved inference speed.
Contribution
The paper presents a novel sequence tagging approach for GEC using Transformers, with custom token transformations and a two-stage fine-tuning process, outperforming previous methods.
Findings
Achieves F0.5 scores of 65.3/66.5 on CoNLL-2014 and 72.4/73.6 on BEA-2019.
Inference speed is up to 10 times faster than seq2seq GEC systems.
Code and models are publicly available.
Abstract
In this paper, we present a simple and efficient GEC sequence tagger using a Transformer encoder. Our system is pre-trained on synthetic data and then fine-tuned in two stages: first on errorful corpora, and second on a combination of errorful and error-free parallel corpora. We design custom token-level transformations to map input tokens to target corrections. Our best single-model/ensemble GEC tagger achieves an of 65.3/66.5 on CoNLL-2014 (test) and of 72.4/73.6 on BEA-2019 (test). Its inference speed is up to 10 times as fast as a Transformer-based seq2seq GEC system. The code and trained models are publicly available.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsLinear Layer · SPEED: Separable Pyramidal Pooling EncodEr-Decoder for Real-Time Monocular Depth Estimation on Low-Resource Settings · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Residual Connection · Label Smoothing · Multi-Head Attention · Adam · *Communicated@Fast*How Do I Communicate to Expedia? · Dropout
