Neural spell-checker: Beyond words with synthetic data generation

TL;DR

This paper introduces two novel Slovene spell-checkers, one traditional and one neural, demonstrating that synthetic data generation and large language models significantly improve spelling correction accuracy.

Contribution

The paper presents and compares a traditional lexicon-based spell-checker and a neural model trained on synthetically generated errors, highlighting the importance of data construction strategies.

Findings

Neural spell-checker outperforms existing methods in Slovene.

Synthetic error data is crucial for training effective neural models.

Large lexicons improve traditional spell-checker performance.

Abstract

Spell-checkers are valuable tools that enhance communication by identifying misspelled words in written texts. Recent improvements in deep learning, and in particular in large language models, have opened new opportunities to improve traditional spell-checkers with new functionalities that not only assess spelling correctness but also the suitability of a word for a given context. In our work, we present and compare two new spell-checkers and evaluate them on synthetic, learner, and more general-domain Slovene datasets. The first spell-checker is a traditional, fast, word-based approach, based on a morphological lexicon with a significantly larger word list compared to existing spell-checkers. The second approach uses a language model trained on a large corpus with synthetically inserted errors. We present the training data construction strategies, which turn out to be a crucial component of neural spell-checkers. Further, the proposed neural model significantly outperforms all existing spell-checkers for Slovene in both precision and recall.