# Learning Topic-Sensitive Word Representations

## TL;DR

This paper introduces two models that learn multiple topic-sensitive word representations using Hierarchical Dirichlet Processes, improving the ability to distinguish word meanings in NLP tasks.

## Contribution

It proposes novel approaches to generate multiple, topic-aware word embeddings, addressing limitations of single representation models.

## Key findings

- Significant improvement in lexical substitution tasks
- Topic-sensitive embeddings outperform traditional single representations
- Models effectively differentiate word meanings based on context

## Abstract

Distributed word representations are widely used for modeling words in NLP tasks. Most of the existing models generate one representation per word and do not consider different meanings of a word. We present two approaches to learn multiple topic-sensitive representations per word by using Hierarchical Dirichlet Process. We observe that by modeling topics and integrating topic distributions for each document we obtain representations that are able to distinguish between different meanings of a given word. Our models yield statistically significant improvements for the lexical substitution task indicating that commonly used single word representations, even when combined with contextual information, are insufficient for this task.