A Universal Part-of-Speech Tagset

TL;DR

This paper introduces a universal part-of-speech tagset and a mapping from various language-specific tagsets, enabling standardized cross-linguistic syntactic analysis and improving unsupervised grammar induction accuracy.

Contribution

It proposes a standardized universal POS tagset and a mapping from 25 different tagsets, facilitating multilingual syntactic research and unsupervised grammar induction.

Findings

Achieved competitive accuracy in unsupervised grammar induction

Created a multilingual dataset with common POS tags for 22 languages

Standardized POS tagging to support cross-linguistic NLP research

Abstract

To facilitate future research in unsupervised induction of syntactic structure and to standardize best-practices, we propose a tagset that consists of twelve universal part-of-speech categories. In addition to the tagset, we develop a mapping from 25 different treebank tagsets to this universal set. As a result, when combined with the original treebank data, this universal tagset and mapping produce a dataset consisting of common parts-of-speech for 22 different languages. We highlight the use of this resource via two experiments, including one that reports competitive accuracies for unsupervised grammar induction without gold standard part-of-speech tags.