A Compositional Framework for Markov Processes

TL;DR

This paper introduces a compositional framework for open Markov processes, formalizing their structure using category theory, and relates their steady-state behavior to principles of minimum dissipation and electrical circuit analogies.

Contribution

It develops a categorical formalism for open Markov processes, linking their behavior to resistor circuits and providing a method to analyze complex systems from simpler components.

Findings

Open Markov processes form a dagger compact category.

Steady states follow a principle of minimum dissipation.

Black box functor maps processes to Lagrangian relations.

Abstract

We define the concept of an "open" Markov process, or more precisely, continuous-time Markov chain, which is one where probability can flow in or out of certain states called "inputs" and "outputs". One can build up a Markov process from smaller open pieces. This process is formalized by making open Markov processes into the morphisms of a dagger compact category. We show that the behavior of a detailed balanced open Markov process is determined by a principle of minimum dissipation, closely related to Prigogine's principle of minimum entropy production. Using this fact, we set up a functor mapping open detailed balanced Markov processes to open circuits made of linear resistors. We also describe how to "black box" an open Markov process, obtaining the linear relation between input and output data that holds in any steady state, including nonequilibrium steady states with a nonzero flow of probability through the system. We prove that black boxing gives a symmetric monoidal dagger functor sending open detailed balanced Markov processes to Lagrangian relations between symplectic vector spaces. This allows us to compute the steady state behavior of an open detailed balanced Markov process from the behaviors of smaller pieces from which it is built. We relate this black box functor to a previously constructed black box functor for circuits.