Copy-composition for Probabilistic Graphical Models

TL;DR

This paper introduces a foundational understanding of copy-composition in probabilistic graphical models, linking it to Bayesian networks and factor graphs, and generalizing probabilistic modeling to dependent types.

Contribution

It provides a theoretical origin story for copy-composition in probabilistic models, connecting it to graphical models and measure kernels, and extends probabilistic modeling to dependent types.

Findings

Copy-composition models Bayesian networks as stochastic terms.

Copy-composition models factor graphs through a double fibration.

Introduces a bifibration of measure kernels for semantics of stochastic terms.

Abstract

In probabilistic modelling, joint distributions are often of more interest than their marginals, but the standard composition of stochastic channels is defined by marginalization. Last year at ACT, the notion of 'copy-composition' was introduced in order to circumvent this problem and express the chain rule of the relative entropy fibrationally, but while that goal was achieved, copy-composition lacked a satisfactory origin story. Here, we supply such a story for two standard probabilistic tools: directed and undirected graphical models. We explain that (directed) Bayesian networks may be understood as ''stochastic terms'' of product type, in which context copy-composition amounts to a pull-push operation. Likewise, we show that (undirected) factor graphs compose by copy-composition. In each case, our construction yields a double fibration of decorated (co)spans. Along the way, we introduce a useful bifibration of measure kernels, to provide semantics for the notion of stochastic term, which allows us to generalize probabilistic modelling from product to dependent types.