Organic Data-Driven Approach for Turkish Grammatical Error Correction and LLMs

TL;DR

This paper presents an organic, data-driven method called clean insertions for Turkish grammatical error correction and improving large language models, achieving state-of-the-art results and demonstrating effectiveness in training.

Contribution

Introduces a novel organic data-driven approach for Turkish grammatical error correction and LLM data cleaning, addressing limitations of synthetic datasets and focusing on Turkish language.

Findings

Achieved state-of-the-art results on Turkish GEC datasets.

Demonstrated effectiveness of the method in training language models.

Improved data quality for Turkish LLM training.

Abstract

Grammatical Error Correction has seen significant progress with the recent advancements in deep learning. As those methods require huge amounts of data, synthetic datasets are being built to fill this gap. Unfortunately, synthetic datasets are not organic enough in some cases and even require clean data to start with. Furthermore, most of the work that has been done is focused mostly on English. In this work, we introduce a new organic data-driven approach, clean insertions, to build parallel Turkish Grammatical Error Correction datasets from any organic data, and to clean the data used for training Large Language Models. We achieve state-of-the-art results on two Turkish Grammatical Error Correction test sets out of the three publicly available ones. We also show the effectiveness of our method on the training losses of training language models.