Modelling Lexical Ambiguity with Density Matrices

TL;DR

This paper introduces neural models that learn density matrices to better represent lexical ambiguity, especially homonymy, within compositional distributional semantics, outperforming existing vector-based models.

Contribution

It presents three novel neural models for learning density matrices from text corpora, enhancing the modeling of lexical ambiguity in compositional semantics.

Findings

Best model outperforms existing vector-based models

Density matrices effectively discriminate between word senses

Neural models improve sense disambiguation in compositional tasks

Abstract

Words can have multiple senses. Compositional distributional models of meaning have been argued to deal well with finer shades of meaning variation known as polysemy, but are not so well equipped to handle word senses that are etymologically unrelated, or homonymy. Moving from vectors to density matrices allows us to encode a probability distribution over different senses of a word, and can also be accommodated within a compositional distributional model of meaning. In this paper we present three new neural models for learning density matrices from a corpus, and test their ability to discriminate between word senses on a range of compositional datasets. When paired with a particular composition method, our best model outperforms existing vector-based compositional models as well as strong sentence encoders.