On Quantum Generalizations of Information-Theoretic Measures and their Contribution to Distributional Semantics

TL;DR

This paper explores quantum generalizations of information-theoretic measures and demonstrates their potential to enhance distributional semantics by modeling word meanings as density operators, capturing richer semantic interactions.

Contribution

It surveys five quantum generalizations of classical measures and shows how they enable more nuanced modeling of semantic phenomena in distributional semantics.

Findings

Quantum measures provide richer semantic modeling

Density operators capture ambiguity and similarity

Enhanced simulation of semantic interactions

Abstract

Information-theoretic measures such as relative entropy and correlation are extremely useful when modeling or analyzing the interaction of probabilistic systems. We survey the quantum generalization of 5 such measures and point out some of their commonalities and interpretations. In particular we find the application of information theory to distributional semantics useful. By modeling the distributional meaning of words as density operators rather than vectors, more of their semantic structure may be exploited. Furthermore, properties of and interactions between words such as ambiguity, similarity and entailment can be simulated more richly and intuitively when using methods from quantum information theory.