Modeling Semantic Relatedness using Global Relation Vectors

TL;DR

This paper introduces a new method for directly learning relation vectors from co-occurrence statistics, enhancing the modeling of semantic relationships beyond existing approaches that manipulate pre-trained vectors.

Contribution

It proposes a variant of GloVe that explicitly connects word vectors with PMI co-occurrence vectors and embeds relation vectors into this space.

Findings

Relation vectors can be learned directly from co-occurrence data.

The method improves modeling of semantic relationships.

It extends GloVe with explicit relation embedding capabilities.

Abstract

Word embedding models such as GloVe rely on co-occurrence statistics from a large corpus to learn vector representations of word meaning. These vectors have proven to capture surprisingly fine-grained semantic and syntactic information. While we may similarly expect that co-occurrence statistics can be used to capture rich information about the relationships between different words, existing approaches for modeling such relationships have mostly relied on manipulating pre-trained word vectors. In this paper, we introduce a novel method which directly learns relation vectors from co-occurrence statistics. To this end, we first introduce a variant of GloVe, in which there is an explicit connection between word vectors and PMI weighted co-occurrence vectors. We then show how relation vectors can be naturally embedded into the resulting vector space.