A Vector Space for Distributional Semantics for Entailment

TL;DR

This paper introduces a vector-space model for distributional semantics that captures semantic entailment, providing a formal foundation and demonstrating improved hyponymy detection in experiments.

Contribution

It develops a formal vector-space framework for entailment in distributional semantics and reinterprets Word2Vec within this framework to predict lexical entailment.

Findings

Substantial improvements in hyponymy detection accuracy

Reinterpretation of Word2Vec as an entailment model

Development of approximate inference procedures for entailment

Abstract

Distributional semantics creates vector-space representations that capture many forms of semantic similarity, but their relation to semantic entailment has been less clear. We propose a vector-space model which provides a formal foundation for a distributional semantics of entailment. Using a mean-field approximation, we develop approximate inference procedures and entailment operators over vectors of probabilities of features being known (versus unknown). We use this framework to reinterpret an existing distributional-semantic model (Word2Vec) as approximating an entailment-based model of the distributions of words in contexts, thereby predicting lexical entailment relations. In both unsupervised and semi-supervised experiments on hyponymy detection, we get substantial improvements over previous results.