Context-aware Stand-alone Neural Spelling Correction

TL;DR

This paper introduces a context-aware neural approach for stand-alone spelling correction that leverages pre-trained language models to jointly detect and correct misspellings, significantly outperforming previous methods.

Contribution

It proposes a novel sequence labeling framework using fine-tuned pre-trained models for stand-alone spelling correction, focusing solely on spelling errors without token insertion or deletion.

Findings

Outperforms previous state-of-the-art by 12.8% absolute F0.5 score

Effectively detects and corrects misspellings using context-aware modeling

Demonstrates the effectiveness of joint detection and correction approach

Abstract

Existing natural language processing systems are vulnerable to noisy inputs resulting from misspellings. On the contrary, humans can easily infer the corresponding correct words from their misspellings and surrounding context. Inspired by this, we address the stand-alone spelling correction problem, which only corrects the spelling of each token without additional token insertion or deletion, by utilizing both spelling information and global context representations. We present a simple yet powerful solution that jointly detects and corrects misspellings as a sequence labeling task by fine-turning a pre-trained language model. Our solution outperforms the previous state-of-the-art result by 12.8% absolute F0.5 score.