Using Multi-Sense Vector Embeddings for Reverse Dictionaries

TL;DR

This paper explores the use of multi-sense vector embeddings in reverse dictionary tasks, demonstrating significant improvements by integrating sense-specific vectors with an attention mechanism.

Contribution

It introduces a method to incorporate multi-sense embeddings into neural networks for reverse dictionaries, enhancing performance over traditional single-sense embeddings.

Findings

Multi-sense embeddings improve reverse dictionary accuracy.

Attention mechanism effectively selects appropriate word senses.

Analysis reveals meaningful sense distributions and attention patterns.

Abstract

Popular word embedding methods such as word2vec and GloVe assign a single vector representation to each word, even if a word has multiple distinct meanings. Multi-sense embeddings instead provide different vectors for each sense of a word. However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word. In this work, we study the effect of multi-sense embeddings on the task of reverse dictionaries. We propose a technique to easily integrate them into an existing neural network architecture using an attention mechanism. Our experiments demonstrate that large improvements can be obtained when employing multi-sense embeddings both in the input sequence as well as for the target representation. An analysis of the sense distributions and of the learned attention is provided as well.