Emergence of Physical Intelligence via Controllable Information Production

TL;DR

This paper introduces Controllable Information Production (CIP), a new framework for intrinsic motivation that links information theory, dynamical systems, and optimal control to foster physical intelligence in agents.

Contribution

CIP provides a principled, dynamical systems-based approach to intrinsic motivation, unifying it with optimal control and demonstrating superior performance in robot learning tasks.

Findings

CIP outperforms prior intrinsic motivation methods on standard benchmarks.

CIP enables agents to solve complex tasks like humanoid self-righting.

CIP reveals a connection between value functions and Kolmogorov-Sinai entropy.

Abstract

Intrinsic Motivation (IM) aims to train agents without external rewards, enabling useful behavior to emerge from the agent's interaction with its environment alone. However, the dominant IM approaches rely on information-theoretic quantities with designer-chosen variables, introducing bias and lacking a principled connection to dynamics or optimal control (OC). We introduce Controllable Information Production (CIP), a new foundation for IM explicitly grounded in dynamical systems and OC. CIP measures the rate at which an agent produces information, capturing controllable complexity without external knowledge or bias. CIP unifies IM and OC into a single framework, formalizing physical intelligence as the control of information production. It further reveals connections between the structure of the value function and Kolmogorov-Sinai entropy. CIP consistently outperforms prior IM methods on standard benchmarks in robot learning and solves tasks they fail on, including humanoid self-righting. These results support a general organizing principle: physical intelligence emerges from driving systems toward the edge of controllable chaos.