# Distributed representation of multi-sense words: A loss-driven approach

## TL;DR

This paper introduces LDMI, a novel model for multi-sense word representation that improves upon traditional single-vector models by clustering word senses to reduce prediction loss, leading to better contextual similarity results.

## Contribution

LDMI is a new loss-driven approach that effectively captures multiple senses of words by clustering occurrences, enhancing distributional representations.

## Key findings

- LDMI outperforms existing methods on contextual word similarity tasks.
- Clustering senses reduces prediction loss for multi-sense words.
- The approach improves multi-sense word representation accuracy.

## Abstract

Word2Vec's Skip Gram model is the current state-of-the-art approach for estimating the distributed representation of words. However, it assumes a single vector per word, which is not well-suited for representing words that have multiple senses. This work presents LDMI, a new model for estimating distributional representations of words. LDMI relies on the idea that, if a word carries multiple senses, then having a different representation for each of its senses should lead to a lower loss associated with predicting its co-occurring words, as opposed to the case when a single vector representation is used for all the senses. After identifying the multi-sense words, LDMI clusters the occurrences of these words to assign a sense to each occurrence. Experiments on the contextual word similarity task show that LDMI leads to better performance than competing approaches.