Semantic Composition via Probabilistic Model Theory

TL;DR

This paper bridges formal semantics and machine learning by interpreting probabilistic graphical models as a probabilistic model theory, enabling improved context-dependent meaning modeling and semantic composition in vector space models.

Contribution

It introduces a novel interpretation of probabilistic graphical models as a probabilistic model theory, enhancing semantic modeling and composition in distributional semantics.

Findings

Improved performance on datasets beyond word similarity.

Explicit modeling of context-dependent meanings.

Enhanced semantic composition techniques.

Abstract

Semantic composition remains an open problem for vector space models of semantics. In this paper, we explain how the probabilistic graphical model used in the framework of Functional Distributional Semantics can be interpreted as a probabilistic version of model theory. Building on this, we explain how various semantic phenomena can be recast in terms of conditional probabilities in the graphical model. This connection between formal semantics and machine learning is helpful in both directions: it gives us an explicit mechanism for modelling context-dependent meanings (a challenge for formal semantics), and also gives us well-motivated techniques for composing distributed representations (a challenge for distributional semantics). We present results on two datasets that go beyond word similarity, showing how these semantically-motivated techniques improve on the performance of vector models.