sense2vec - A Fast and Accurate Method for Word Sense Disambiguation In Neural Word Embeddings

TL;DR

sense2vec introduces a fast, supervised method for word sense disambiguation in neural embeddings, improving accuracy and efficiency for NLP tasks across multiple languages and nuanced senses.

Contribution

It presents a novel supervised approach for modeling multiple word senses in embeddings, addressing efficiency and application challenges of prior methods.

Findings

Disambiguates contrastive and nuanced senses effectively.

Achieves over 8% error reduction in dependency parsing.

Demonstrates broad applicability across languages.

Abstract

Neural word representations have proven useful in Natural Language Processing (NLP) tasks due to their ability to efficiently model complex semantic and syntactic word relationships. However, most techniques model only one representation per word, despite the fact that a single word can have multiple meanings or "senses". Some techniques model words by using multiple vectors that are clustered based on context. However, recent neural approaches rarely focus on the application to a consuming NLP algorithm. Furthermore, the training process of recent word-sense models is expensive relative to single-sense embedding processes. This paper presents a novel approach which addresses these concerns by modeling multiple embeddings for each word based on supervised disambiguation, which provides a fast and accurate way for a consuming NLP model to select a sense-disambiguated embedding. We demonstrate that these embeddings can disambiguate both contrastive senses such as nominal and verbal senses as well as nuanced senses such as sarcasm. We further evaluate Part-of-Speech disambiguated embeddings on neural dependency parsing, yielding a greater than 8% average error reduction in unlabeled attachment scores across 6 languages.