Approximate Inference via Fibrations of Statistical Games

TL;DR

This paper presents a novel categorical framework for approximate inference in statistical models, using fibrations of statistical games and Bayesian lenses to formalize compositional structures and loss functions.

Contribution

It introduces the concept of 2-fibrations of statistical games and coparameterized Bayesian lenses, providing a new formalization of approximate inference with compositional properties.

Findings

Relative entropy as a strict section in the fibration

Chain rule formalized by horizontal composition

Coparameterized Bayesian updates compose optically

Abstract

We characterize a number of well known systems of approximate inference as loss models: lax sections of 2-fibrations of statistical games, constructed by attaching internally-defined loss functions to Bayesian lenses. Our examples include the relative entropy, which constitutes a strict section, and whose chain rule is formalized by the horizontal composition of the 2-fibration. In order to capture this compositional structure, we first introduce the notion of 'copy-composition', alongside corresponding bicategories through which the composition of copy-discard categories factorizes. These bicategories are a variant of the Copara construction, and so we additionally introduce coparameterized Bayesian lenses, proving that coparameterized Bayesian updates compose optically, as in the non-coparameterized case.