Adapting LLMs for Minimal-edit Grammatical Error Correction

TL;DR

This paper enhances large language models for minimal-edit grammatical error correction by proposing a novel training schedule, achieving state-of-the-art results, and analyzing dataset impacts.

Contribution

It introduces a new training schedule for LLMs tailored to minimal-edit GEC and evaluates the effects of dataset detokenization and error correction on performance.

Findings

Achieved new state-of-the-art on BEA-test with a single model

Detokenization of datasets impacts GEC results

Corrected erroneous examples improve training effectiveness

Abstract

Decoder-only large language models have shown superior performance in the fluency-edit English Grammatical Error Correction, but their adaptation for minimal-edit English GEC is still underexplored. To improve their effectiveness in the minimal-edit approach, we explore the error rate adaptation topic and propose a novel training schedule method. Our experiments set a new state-of-the-art result for a single-model system on the BEA-test set. We also detokenize the most common English GEC datasets to match the natural way of writing text. During the process, we find that there are errors in them. Our experiments analyze whether training on detokenized datasets impacts the results and measure the impact of the usage of the datasets with corrected erroneous examples. To facilitate reproducibility, we have released the source code used to train our models.