A categorical characterization of relative entropy on standard Borel spaces

TL;DR

This paper provides a categorical framework for understanding relative entropy on standard Borel spaces, defining it as a functor and establishing key properties like convexity and lower semicontinuity.

Contribution

It introduces a categorical approach to relative entropy on standard Borel spaces, including a new category for statistical inference and a functorial definition of relative entropy.

Findings

Relative entropy is characterized as a functor into Lawvere's category.

The paper proves convexity and lower semicontinuity of the relative entropy functor.

It establishes the uniqueness of the categorical characterization.

Abstract

We give a categorical treatment, in the spirit of Baez and Fritz, of relative entropy for probability distributions defined on standard Borel spaces. We define a category suitable for reasoning about statistical inference on standard Borel spaces. We define relative entropy as a functor into Lawvere's category and we show convexity, lower semicontinuity and uniqueness.