Intrinsic Motivation in Dynamical Control Systems

TL;DR

This paper explores an information-theoretic framework for intrinsic motivation in control systems, using empowerment to guide autonomous behavior without explicit rewards, and demonstrates its effectiveness on benchmark problems.

Contribution

It introduces a novel empowerment-based approach to intrinsic motivation, providing a computationally efficient algorithm and linking behavior to system dynamics.

Findings

Successful application on benchmark control problems

Empowerment guides intrinsic behaviors effectively

Links between information-theoretic control and system dynamics

Abstract

Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation. The computational principles underlying this behavior remain poorly understood. In this study, we investigate an information-theoretic approach to intrinsic motivation, based on maximizing an agent's empowerment (the mutual information between its past actions and future states). We show that this approach generalizes previous attempts to formalize intrinsic motivation, and we provide a computationally efficient algorithm for computing the necessary quantities. We test our approach on several benchmark control problems, and we explain its success in guiding intrinsically motivated behaviors by relating our information-theoretic control function to fundamental properties of the dynamical system representing the combined agent-environment system. This opens the door for designing practical artificial, intrinsically motivated controllers and for linking animal behaviors to their dynamical properties.