Using Multi-Sense Vector Embeddings for Reverse Dictionaries
Michael A. Hedderich, Andrew Yates, Dietrich Klakow, Gerard de Melo

TL;DR
This paper explores the use of multi-sense vector embeddings in reverse dictionary tasks, demonstrating significant improvements by integrating sense-specific vectors with an attention mechanism.
Contribution
It introduces a method to incorporate multi-sense embeddings into neural networks for reverse dictionaries, enhancing performance over traditional single-sense embeddings.
Findings
Multi-sense embeddings improve reverse dictionary accuracy.
Attention mechanism effectively selects appropriate word senses.
Analysis reveals meaningful sense distributions and attention patterns.
Abstract
Popular word embedding methods such as word2vec and GloVe assign a single vector representation to each word, even if a word has multiple distinct meanings. Multi-sense embeddings instead provide different vectors for each sense of a word. However, they typically cannot serve as a drop-in replacement for conventional single-sense embeddings, because the correct sense vector needs to be selected for each word. In this work, we study the effect of multi-sense embeddings on the task of reverse dictionaries. We propose a technique to easily integrate them into an existing neural network architecture using an attention mechanism. Our experiments demonstrate that large improvements can be obtained when employing multi-sense embeddings both in the input sequence as well as for the target representation. An analysis of the sense distributions and of the learned attention is provided as well.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNatural Language Processing Techniques · Topic Modeling · Text and Document Classification Technologies
MethodsGloVe Embeddings
