Optimal measures and Markov transition kernels

TL;DR

This paper generalizes classical variational problems in information theory by studying optimal measures and Markov kernels, revealing conditions under which deterministic transitions are sub-optimal and characterizing families of optimal measures.

Contribution

It introduces a framework for understanding optimal measures beyond exponential families, emphasizing the role of convex duality and mutual absolute continuity in Markov kernels.

Findings

Optimal measures are characterized by convex duality properties.

Deterministic Markov kernels are generally sub-optimal under information constraints.

Examples show deterministic kernels can lead to unbounded error or infinite information communication.

Abstract

We study optimal solutions to an abstract optimization problem for measures, which is a generalization of classical variational problems in information theory and statistical physics. In the classical problems, information and relative entropy are defined using the Kullback-Leibler divergence, and for this reason optimal measures belong to a one-parameter exponential family. Measures within such a family have the property of mutual absolute continuity. Here we show that this property characterizes other families of optimal positive measures if a functional representing information has a strictly convex dual. Mutual absolute continuity of optimal probability measures allows us to strictly separate deterministic and non-deterministic Markov transition kernels, which play an important role in theories of decisions, estimation, control, communication and computation. We show that deterministic transitions are strictly sub-optimal, unless information resource with a strictly convex dual is unconstrained. For illustration, we construct an example where, unlike non-deterministic, any deterministic kernel either has negatively infinite expected utility (unbounded expected error) or communicates infinite information.