Redefining part-of-speech classes with distributional semantic models

TL;DR

This study explores how word embeddings can reveal and redefine part-of-speech boundaries, uncovering inconsistencies and supporting graded PoS classifications across languages.

Contribution

It demonstrates that distributional semantic models contain rich PoS information, enabling the discovery of nuanced and potentially inconsistent PoS groupings.

Findings

PoS information is distributed across multiple vector components

Embeddings can identify hidden inconsistencies in PoS annotation

Supports the concept of graded or soft PoS affiliations

Abstract

This paper studies how word embeddings trained on the British National Corpus interact with part of speech boundaries. Our work targets the Universal PoS tag set, which is currently actively being used for annotation of a range of languages. We experiment with training classifiers for predicting PoS tags for words based on their embeddings. The results show that the information about PoS affiliation contained in the distributional vectors allows us to discover groups of words with distributional patterns that differ from other words of the same part of speech. This data often reveals hidden inconsistencies of the annotation process or guidelines. At the same time, it supports the notion of `soft' or `graded' part of speech affiliations. Finally, we show that information about PoS is distributed among dozens of vector components, not limited to only one or two features.