Neural Network Translation Models for Grammatical Error Correction

TL;DR

This paper introduces neural network models for grammatical error correction, improving upon traditional phrase-based SMT systems by leveraging continuous word representations and contextual information for better accuracy.

Contribution

It proposes neural network global lexicon and joint models that address SMT limitations, enhancing GEC performance with non-linear mappings and context utilization.

Findings

Significant accuracy improvement over state-of-the-art GEC systems

Neural models outperform phrase-based SMT in error correction

Better generalization through continuous word representations

Abstract

Phrase-based statistical machine translation (SMT) systems have previously been used for the task of grammatical error correction (GEC) to achieve state-of-the-art accuracy. The superiority of SMT systems comes from their ability to learn text transformations from erroneous to corrected text, without explicitly modeling error types. However, phrase-based SMT systems suffer from limitations of discrete word representation, linear mapping, and lack of global context. In this paper, we address these limitations by using two different yet complementary neural network models, namely a neural network global lexicon model and a neural network joint model. These neural networks can generalize better by using continuous space representation of words and learn non-linear mappings. Moreover, they can leverage contextual information from the source sentence more effectively. By adding these two components, we achieve statistically significant improvement in accuracy for grammatical error correction over a state-of-the-art GEC system.