Refining Czech GEC: Insights from a Multi-Experiment Approach

TL;DR

This paper introduces a state-of-the-art Czech grammar error correction system based on a Transformer neural network with synthetic data augmentation, extensive experiments, and evaluation of large language models, achieving high performance and efficiency.

Contribution

It presents a novel real-time synthetic error generation pipeline and comprehensive experimental analysis for Czech GEC, improving over previous methods.

Findings

The system achieves state-of-the-art accuracy for Czech GEC.

Synthetic error augmentation improves model performance.

Large language models show competitive results in Czech GEC.

Abstract

We present a grammar error correction (GEC) system that achieves state of the art for the Czech language. Our system is based on a neural network translation approach with the Transformer architecture, and its key feature is its real-time synthetic generation pipeline, which dynamically augments sentences with artificial errors by introducing both language-agnostic and Czech-specific errors. We conduct a comprehensive series of experiments, investigating the Czech GEC corpora as bases for synthetic error introduction, several error generation strategies, domain balancing, tokenization granularity, model size, and data scaling during fine-tuning. Additionally, we evaluate the performance of large language models (LLMs) on Czech GEC in both end-user and expert fine-tuning scenarios. Our best-performing model is superior both in performance and computational efficiency. The source code and the trained model links are available on https://github.com/ufal/tsd2025-gec.