Conserved active information

TL;DR

This paper introduces a new measure called conserved active information, which quantifies net information change across search spaces, revealing regimes hidden from traditional divergence measures and resolving critiques of active information.

Contribution

The paper proposes conserved active information $I^igoplus$, extending active information to account for net information gain/loss respecting conservation laws, with formal proofs and diverse applications.

Findings

$I^igoplus$ reveals regimes where strong knowledge reduces disorder.

Formal proof distinguishes disorder regimes under uniform baseline.

Applications demonstrated in Markov chains and cosmology.

Abstract

We introduce conserved active information $I^\oplus$, a symmetric extension of active information that quantifies net information gain/loss across the entire search space, respecting No-Free-Lunch conservation. Through Bernoulli and uniform-baseline examples, we show $I^\oplus$ reveals regimes hidden from KL divergence, such as when strong knowledge reduces global disorder. Such regimes are proven formally under uniform baseline, distinguishing disorder (increasing mild knowledge from order-imposing strong knowledge. We further illustrate these regimes with examples from Markov chains and cosmological fine-tuning. This resolves a longstanding critique of active information while enabling applications in search, optimization, and beyond.