Compositional Demographic Word Embeddings

TL;DR

This paper introduces a method for creating personalized demographic-aware word embeddings by composing demographic information, improving language modeling and word association tasks, while discussing ethical considerations.

Contribution

It proposes a novel compositional approach to generate demographic-specific word embeddings from limited demographic data, enhancing personalization in NLP applications.

Findings

Demographic embeddings outperform generic embeddings in language modeling.

Demographic embeddings improve word association task performance.

Trade-offs exist between the number of demographic attributes used and their effectiveness.

Abstract

Word embeddings are usually derived from corpora containing text from many individuals, thus leading to general purpose representations rather than individually personalized representations. While personalized embeddings can be useful to improve language model performance and other language processing tasks, they can only be computed for people with a large amount of longitudinal data, which is not the case for new users. We propose a new form of personalized word embeddings that use demographic-specific word representations derived compositionally from full or partial demographic information for a user (i.e., gender, age, location, religion). We show that the resulting demographic-aware word representations outperform generic word representations on two tasks for English: language modeling and word associations. We further explore the trade-off between the number of available attributes and their relative effectiveness and discuss the ethical implications of using them.