CUNI System for the Building Educational Applications 2019 Shared Task: Grammatical Error Correction
Jakub N\'aplava, Milan Straka

TL;DR
This paper presents Transformer-based neural machine translation systems for grammatical error correction in educational applications, with various enhancements and training strategies, achieving competitive results across different data tracks.
Contribution
The authors introduce improved Transformer models with novel training techniques for grammatical error correction, demonstrating effectiveness across multiple data scenarios.
Findings
Achieved 59.39 F0.5 score in the Restricted Track.
Reached 44.13 F0.5 score in the Low-Resource Track.
Obtained 64.55 F0.5 score after fine-tuning, ranking third in the Unrestricted Track.
Abstract
In this paper, we describe our systems submitted to the Building Educational Applications (BEA) 2019 Shared Task (Bryant et al., 2019). We participated in all three tracks. Our models are NMT systems based on the Transformer model, which we improve by incorporating several enhancements: applying dropout to whole source and target words, weighting target subwords, averaging model checkpoints, and using the trained model iteratively for correcting the intermediate translations. The system in the Restricted Track is trained on the provided corpora with oversampled "cleaner" sentences and reaches 59.39 F0.5 score on the test set. The system in the Low-Resource Track is trained from Wikipedia revision histories and reaches 44.13 F0.5 score. Finally, we finetune the system from the Low-Resource Track on restricted data and achieve 64.55 F0.5 score, placing third in the Unrestricted Track.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Text Readability and Simplification
MethodsLinear Layer · Absolute Position Encodings · Position-Wise Feed-Forward Layer · Residual Connection · Byte Pair Encoding · Dense Connections · Label Smoothing · *Communicated@Fast*How Do I Communicate to Expedia? · Adam · Softmax
