Distributional Inclusion Hypothesis and Quantifications: Probing for Hypernymy in Functional Distributional Semantics

TL;DR

This paper investigates how Functional Distributional Semantics models learn hypernymy, revealing that they do so under certain conditions related to the Distributional Inclusion Hypothesis, and proposes a new training objective to enhance this learning.

Contribution

The paper connects hypernymy learning in FDS models with the DIH and introduces a novel training objective that improves hypernymy detection, especially in challenging cases.

Findings

FDS models learn hypernymy when trained on data following DIH.

A new training objective enables hypernymy learning under reverse DIH.

Improved hypernymy detection from real corpora.

Abstract

Functional Distributional Semantics (FDS) models the meaning of words by truth-conditional functions. This provides a natural representation for hypernymy but no guarantee that it can be learnt when FDS models are trained on a corpus. In this paper, we probe into FDS models and study the representations learnt, drawing connections between quantifications, the Distributional Inclusion Hypothesis (DIH), and the variational-autoencoding objective of FDS model training. Using synthetic data sets, we reveal that FDS models learn hypernymy on a restricted class of corpus that strictly follows the DIH. We further introduce a training objective that both enables hypernymy learning under the reverse of the DIH and improves hypernymy detection from real corpora.