Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection

TL;DR

This paper introduces DIVE, an unsupervised, low-dimensional embedding method for hypernymy detection that significantly outperforms previous approaches in accuracy and efficiency across multiple datasets.

Contribution

The paper presents DIVE, a novel unsupervised embedding technique that captures hypernymy relations with high accuracy using interpretable, compact vectors.

Findings

DIVE achieves up to double the precision of previous methods.

It provides the highest average performance across 11 datasets.

DIVE yields many new state-of-the-art results.

Abstract

Modeling hypernymy, such as poodle is-a dog, is an important generalization aid to many NLP tasks, such as entailment, coreference, relation extraction, and question answering. Supervised learning from labeled hypernym sources, such as WordNet, limits the coverage of these models, which can be addressed by learning hypernyms from unlabeled text. Existing unsupervised methods either do not scale to large vocabularies or yield unacceptably poor accuracy. This paper introduces distributional inclusion vector embedding (DIVE), a simple-to-implement unsupervised method of hypernym discovery via per-word non-negative vector embeddings which preserve the inclusion property of word contexts in a low-dimensional and interpretable space. In experimental evaluations more comprehensive than any previous literature of which we are aware-evaluating on 11 datasets using multiple existing as well as newly proposed scoring functions-we find that our method provides up to double the precision of previous unsupervised embeddings, and the highest average performance, using a much more compact word representation, and yielding many new state-of-the-art results.