Probabilistic Relation Induction in Vector Space Embeddings

TL;DR

This paper introduces two probabilistic models to improve the extraction of relational knowledge from word embeddings, enhancing accuracy and interpretability of relation induction.

Contribution

It presents novel probabilistic models for relation induction in word embeddings, outperforming existing methods and clarifying what relational information can be reliably extracted.

Findings

Models achieve more accurate relation predictions

Probabilistic approach offers better interpretability

Outperforms existing relation induction methods

Abstract

Word embeddings have been found to capture a surprisingly rich amount of syntactic and semantic knowledge. However, it is not yet sufficiently well-understood how the relational knowledge that is implicitly encoded in word embeddings can be extracted in a reliable way. In this paper, we propose two probabilistic models to address this issue. The first model is based on the common relations-as-translations view, but is cast in a probabilistic setting. Our second model is based on the much weaker assumption that there is a linear relationship between the vector representations of related words. Compared to existing approaches, our models lead to more accurate predictions, and they are more explicit about what can and cannot be extracted from the word embedding.