Applying Winnow to Context-Sensitive Spelling Correction

TL;DR

This paper demonstrates that the Winnow algorithm, a multiplicative weight-updating method, effectively improves context-sensitive spelling correction, especially with full feature sets and in adapting to unfamiliar test data, outperforming Bayesian classifiers.

Contribution

The paper introduces the application of Winnow to spelling correction, showing its advantages over Bayesian methods with full features and in adapting to new test environments.

Findings

Winnow performs comparably to Bayesian methods with pruned features.

Winnow outperforms Bayesian classifiers with full feature sets.

Winnow adapts better to dissimilar test sets using combined learning strategies.

Abstract

Multiplicative weight-updating algorithms such as Winnow have been studied extensively in the COLT literature, but only recently have people started to use them in applications. In this paper, we apply a Winnow-based algorithm to a task in natural language: context-sensitive spelling correction. This is the task of fixing spelling errors that happen to result in valid words, such as substituting {\it to\/} for {\it too}, {\it casual\/} for {\it causal}, and so on. Previous approaches to this problem have been statistics-based; we compare Winnow to one of the more successful such approaches, which uses Bayesian classifiers. We find that: (1)~When the standard (heavily-pruned) set of features is used to describe problem instances, Winnow performs comparably to the Bayesian method; (2)~When the full (unpruned) set of features is used, Winnow is able to exploit the new features and convincingly outperform Bayes; and (3)~When a test set is encountered that is dissimilar to the training set, Winnow is better than Bayes at adapting to the unfamiliar test set, using a strategy we will present for combining learning on the training set with unsupervised learning on the (noisy) test set.