Do Multi-Sense Embeddings Improve Natural Language Understanding?

TL;DR

This paper investigates whether multi-sense embeddings enhance natural language understanding by testing a Chinese Restaurant Process-based model across various NLP tasks, revealing task-dependent benefits and emphasizing real-world application testing.

Contribution

Introduces a multi-sense embedding model using Chinese Restaurant Processes and evaluates its impact on multiple NLP tasks with a focus on real application performance.

Findings

Improves performance on part-of-speech tagging, semantic relation identification, and semantic relatedness.

Does not improve named entity recognition or sentiment analysis.

Highlights the importance of task-specific evaluation of embedding models.

Abstract

Learning a distinct representation for each sense of an ambiguous word could lead to more powerful and fine-grained models of vector-space representations. Yet while `multi-sense' methods have been proposed and tested on artificial word-similarity tasks, we don't know if they improve real natural language understanding tasks. In this paper we introduce a multi-sense embedding model based on Chinese Restaurant Processes that achieves state of the art performance on matching human word similarity judgments, and propose a pipelined architecture for incorporating multi-sense embeddings into language understanding. We then test the performance of our model on part-of-speech tagging, named entity recognition, sentiment analysis, semantic relation identification and semantic relatedness, controlling for embedding dimensionality. We find that multi-sense embeddings do improve performance on some tasks (part-of-speech tagging, semantic relation identification, semantic relatedness) but not on others (named entity recognition, various forms of sentiment analysis). We discuss how these differences may be caused by the different role of word sense information in each of the tasks. The results highlight the importance of testing embedding models in real applications.