Emergence of Implicit World Models from Mortal Agents

TL;DR

This paper explores how autonomous agents can develop implicit world models and active exploration behaviors as emergent properties of open-ended behavior optimization, inspired by biological systems and meta-reinforcement learning.

Contribution

It proposes a hypothetical architecture combining meta-reinforcement learning and homeostasis to enable agents to implicitly acquire world models and explore actively.

Findings

Discussion of biological principles like homeostasis as motivation

Theoretical framework for implicit world model emergence

Proposed architecture for autonomous exploration

Abstract

We discuss the possibility of world models and active exploration as emergent properties of open-ended behavior optimization in autonomous agents. In discussing the source of the open-endedness of living things, we start from the perspective of biological systems as understood by the mechanistic approach of theoretical biology and artificial life. From this perspective, we discuss the potential of homeostasis in particular as an open-ended objective for autonomous agents and as a general, integrative extrinsic motivation. We then discuss the possibility of implicitly acquiring a world model and active exploration through the internal dynamics of a network, and a hypothetical architecture for this, by combining meta-reinforcement learning, which assumes domain adaptation as a system that achieves robust homeostasis.