Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correction

TL;DR

This paper demonstrates that a phrase-based statistical machine translation system, with optimized parameter tuning, achieves state-of-the-art results in automatic grammatical error correction, surpassing previous methods significantly.

Contribution

The authors introduce a tuning approach for GEC that outperforms existing results, including the integration of dense and sparse features for further improvements.

Findings

Achieved 49.49% M^2 score on CoNLL-2014 test set

Bare-bones SMT with tuning outperforms previous neural-feature SMT systems

Parameter tuning interactions significantly impact GEC performance

Abstract

In this work, we study parameter tuning towards the M^2 metric, the standard metric for automatic grammar error correction (GEC) tasks. After implementing M^2 as a scorer in the Moses tuning framework, we investigate interactions of dense and sparse features, different optimizers, and tuning strategies for the CoNLL-2014 shared task. We notice erratic behavior when optimizing sparse feature weights with M^2 and offer partial solutions. We find that a bare-bones phrase-based SMT setup with task-specific parameter-tuning outperforms all previously published results for the CoNLL-2014 test set by a large margin (46.37% M^2 over previously 41.75%, by an SMT system with neural features) while being trained on the same, publicly available data. Our newly introduced dense and sparse features widen that gap, and we improve the state-of-the-art to 49.49% M^2.