Iterative Scaling Algorithm for Channels

TL;DR

This paper introduces an iterative scaling algorithm for channels, enabling the evaluation of KL-projections crucial for analyzing information interactions, synergy, and complexity in systems.

Contribution

It generalizes the iterative scaling algorithm from probability distributions to channels, facilitating advanced information-theoretic analyses.

Findings

Provides a method for KL-projection evaluation of channels

Enables decomposition of mutual information and analysis of interactions

Extends iterative scaling to transition kernels

Abstract

Here we define a procedure for evaluating KL-projections (I- and rI-projections) of channels. These can be useful in the decomposition of mutual information between input and outputs, e.g. to quantify synergies and interactions of different orders, as well as information integration and other related measures of complexity. The algorithm is a generalization of the standard iterative scaling algorithm, which we here extend from probability distributions to channels (also known as transition kernels).