Ordering information on distributions

TL;DR

This thesis introduces a class of partial orders on probability distributions and density operators to model information content, exploring theoretical foundations and potential applications in computational linguistics.

Contribution

It defines a new framework of information orderings on distributions and density operators, linking order theory with information content and language modeling.

Findings

Defined a family of information orderings on probability distributions.

Extended the concept to density operators, including the maximum eigenvalue order.

Discussed potential applications in language models for entailment and disambiguation.

Abstract

This thesis details a class of partial orders on the space of probability distributions and the space of density operators which capture the idea of information content. Some links to domain theory and computational linguistics are also discussed. Chapter 1 details some useful theorems from order theory. In Chapter 2 we define a notion of an information ordering on the space of probability distributions and see that this gives rise to a large class of orderings. In Chapter 3 we extend the idea of an information ordering to the space of density operators and in particular look at the maximum eigenvalue order. We will discuss whether this order might be unique given certain restrictions. In Chapter 4 we discuss a possible application in distributional language models, namely in the study of entailment and disambiguation.