Exploring the Value of Personalized Word Embeddings

TL;DR

This paper introduces personalized word embeddings for language modeling, demonstrating that combining personalized and generic embeddings improves perplexity and aids authorship attribution.

Contribution

It proposes a novel approach to personalized word embeddings and shows their effectiveness in improving language model performance and authorship attribution.

Findings

Combining personalized and generic embeddings reduces perplexity by 4.7%.

Personalized embeddings vary more for words in specific psycholinguistic categories.

Personalized embeddings enhance authorship attribution accuracy.

Abstract

In this paper, we introduce personalized word embeddings, and examine their value for language modeling. We compare the performance of our proposed prediction model when using personalized versus generic word representations, and study how these representations can be leveraged for improved performance. We provide insight into what types of words can be more accurately predicted when building personalized models. Our results show that a subset of words belonging to specific psycholinguistic categories tend to vary more in their representations across users and that combining generic and personalized word embeddings yields the best performance, with a 4.7% relative reduction in perplexity. Additionally, we show that a language model using personalized word embeddings can be effectively used for authorship attribution.