Grammatical Error Correction as GAN-like Sequence Labeling

TL;DR

This paper introduces a GAN-inspired sequence labeling model for grammatical error correction that improves training by better mimicking real error distributions, leading to state-of-the-art results.

Contribution

It proposes a novel GAN-like framework with a discriminator and generator for GEC, addressing training-inference mismatch and enhancing correction accuracy.

Findings

Achieves state-of-the-art performance on multiple benchmarks.

Sampling from real error distributions improves correction quality.

The model effectively reduces training-inference mismatch.

Abstract

In Grammatical Error Correction (GEC), sequence labeling models enjoy fast inference compared to sequence-to-sequence models; however, inference in sequence labeling GEC models is an iterative process, as sentences are passed to the model for multiple rounds of correction, which exposes the model to sentences with progressively fewer errors at each round. Traditional GEC models learn from sentences with fixed error rates. Coupling this with the iterative correction process causes a mismatch between training and inference that affects final performance. In order to address this mismatch, we propose a GAN-like sequence labeling model, which consists of a grammatical error detector as a discriminator and a grammatical error labeler with Gumbel-Softmax sampling as a generator. By sampling from real error distributions, our errors are more genuine compared to traditional synthesized GEC errors, thus alleviating the aforementioned mismatch and allowing for better training. Our results on several evaluation benchmarks demonstrate that our proposed approach is effective and improves the previous state-of-the-art baseline.