Incorporating Both Distributional and Relational Semantics in Word Representations

TL;DR

This paper explores combining distributional and relational semantics in word representations using ADMM, showing potential improvements in knowledge base tasks, analogy tests, and parsing.

Contribution

It introduces a novel method to jointly optimize distributional and relational semantics in word embeddings using ADMM.

Findings

Improved performance in knowledge base completion

Enhanced results on analogy tests

Some gains in parsing accuracy

Abstract

We investigate the hypothesis that word representations ought to incorporate both distributional and relational semantics. To this end, we employ the Alternating Direction Method of Multipliers (ADMM), which flexibly optimizes a distributional objective on raw text and a relational objective on WordNet. Preliminary results on knowledge base completion, analogy tests, and parsing show that word representations trained on both objectives can give improvements in some cases.