ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction

TL;DR

This paper presents ChatLang-8, a framework leveraging large language models to generate diverse, high-quality synthetic data for grammatical error correction, significantly improving model performance and dataset variability.

Contribution

The paper introduces a novel automated framework and a new dataset, ChatLang-8, for generating diverse GEC data using LLMs, enhancing data quality and model training.

Findings

ChatLang-8 has 1 million human-like error pairs.

Models trained on ChatLang-8 outperform those trained on existing datasets.

ChatLang-8 exhibits more uniform pattern diversity.

Abstract

We explore and improve the capabilities of LLMs to generate data for grammatical error correction (GEC). When merely producing parallel sentences, their patterns are too simplistic to be valuable as a corpus. To address this issue, we propose an automated framework that includes a Subject Selector, Grammar Selector, Prompt Manager, and Evaluator. Additionally, we introduce a new dataset for GEC tasks, named ChatLang-8, which encompasses eight types of subject nouns and 23 types of grammar. It consists of 1 million pairs featuring human-like grammatical errors. Our experiments reveal that ChatLang-8 exhibits a more uniform pattern composition compared to existing GEC datasets. Furthermore, we observe improved model performance when using ChatLang-8 instead of existing GEC datasets. The experimental results suggest that our framework and ChatLang-8 are valuable resources for enhancing ChatGPT's data generation capabilities.