Multi-sense embeddings through a word sense disambiguation process

TL;DR

This paper introduces a novel unsupervised method called MSSA for disambiguating words by sense, enhancing semantic representations with multi-sense embeddings, and demonstrates its effectiveness across multiple benchmarks.

Contribution

The paper presents an unsupervised disambiguation technique, a multi-sense embedding model adaptable to existing algorithms, and a reusable methodology for refining semantic representations.

Findings

Achieved state-of-the-art results on six word similarity benchmarks.

Outperformed several complex existing systems in semantic tasks.

Provided a flexible approach applicable to various embedding algorithms.

Abstract

Natural Language Understanding has seen an increasing number of publications in the last few years, especially after robust word embeddings models became prominent, when they proved themselves able to capture and represent semantic relationships from massive amounts of data. Nevertheless, traditional models often fall short in intrinsic issues of linguistics, such as polysemy and homonymy. Any expert system that makes use of natural language in its core, can be affected by a weak semantic representation of text, resulting in inaccurate outcomes based on poor decisions. To mitigate such issues, we propose a novel approach called Most Suitable Sense Annotation (MSSA), that disambiguates and annotates each word by its specific sense, considering the semantic effects of its context. Our approach brings three main contributions to the semantic representation scenario: (i) an unsupervised technique that disambiguates and annotates words by their senses, (ii) a multi-sense embeddings model that can be extended to any traditional word embeddings algorithm, and (iii) a recurrent methodology that allows our models to be re-used and their representations refined. We test our approach on six different benchmarks for the word similarity task, showing that our approach can produce state-of-the-art results and outperforms several more complex state-of-the-art systems.