Learning to Resolve Natural Language Ambiguities: A Unified Approach

TL;DR

This paper introduces a unified, data-driven approach using a sparse network of linear separators and the Winnow algorithm for natural language disambiguation, demonstrating superior or comparable performance across multiple lexical tasks.

Contribution

It proposes a novel attribute-efficient learning method based on linear separators that outperforms or matches existing approaches in language ambiguity resolution tasks.

Findings

Outperforms existing methods in lexical disambiguation tasks

Effective in high-dimensional attribute spaces

Applicable across various language processing tasks

Abstract

We analyze a few of the commonly used statistics based and machine learning algorithms for natural language disambiguation tasks and observe that they can be re-cast as learning linear separators in the feature space. Each of the methods makes a priori assumptions, which it employs, given the data, when searching for its hypothesis. Nevertheless, as we show, it searches a space that is as rich as the space of all linear separators. We use this to build an argument for a data driven approach which merely searches for a good linear separator in the feature space, without further assumptions on the domain or a specific problem. We present such an approach - a sparse network of linear separators, utilizing the Winnow learning algorithm - and show how to use it in a variety of ambiguity resolution problems. The learning approach presented is attribute-efficient and, therefore, appropriate for domains having very large number of attributes. In particular, we present an extensive experimental comparison of our approach with other methods on several well studied lexical disambiguation tasks such as context-sensitive spelling correction, prepositional phrase attachment and part of speech tagging. In all cases we show that our approach either outperforms other methods tried for these tasks or performs comparably to the best.