# A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning

## TL;DR

This paper presents a neural grammatical error correction system that leverages improved pre-training and sequential transfer learning on generated noisy data, achieving competitive results in low-resource settings.

## Contribution

It introduces a novel approach combining noising-based data augmentation with sequential transfer learning for grammatical error correction.

## Key findings

- Achieves competitive results in low-resource and restricted tracks
- Utilizes generated noisy data for effective pre-training
- Demonstrates the effectiveness of transfer learning in domain adaptation

## Abstract

Grammatical error correction can be viewed as a low-resource sequence-to-sequence task, because publicly available parallel corpora are limited. To tackle this challenge, we first generate erroneous versions of large unannotated corpora using a realistic noising function. The resulting parallel corpora are subsequently used to pre-train Transformer models. Then, by sequentially applying transfer learning, we adapt these models to the domain and style of the test set. Combined with a context-aware neural spellchecker, our system achieves competitive results in both restricted and low resource tracks in ACL 2019 BEA Shared Task. We release all of our code and materials for reproducibility.