Interpretable Word Embeddings via Informative Priors

TL;DR

This paper introduces a method for creating interpretable word embeddings by incorporating informative priors, enhancing interpretability and domain relevance without sacrificing performance.

Contribution

It proposes a novel approach using informative priors to produce interpretable, domain-informed word embeddings, addressing interpretability issues in traditional unsupervised models.

Findings

Sensible priors improve semantic concept capture.

The method matches or exceeds state-of-the-art performance.

Retains simplicity and generalizability of probabilistic models.

Abstract

Word embeddings have demonstrated strong performance on NLP tasks. However, lack of interpretability and the unsupervised nature of word embeddings have limited their use within computational social science and digital humanities. We propose the use of informative priors to create interpretable and domain-informed dimensions for probabilistic word embeddings. Experimental results show that sensible priors can capture latent semantic concepts better than or on-par with the current state of the art, while retaining the simplicity and generalizability of using priors.