Intrinsically Motivated Learning of Causal World Models

TL;DR

This paper investigates how intrinsically motivated learning of causal world models, through active interventions, can enhance generalization and transfer in reinforcement learning, aiming to emulate human-like incremental knowledge building.

Contribution

It proposes that actively selecting actions to gather interventional data improves causal inference in world models, advancing the development of more general and adaptable AI systems.

Findings

Active interventions improve causal structure learning.

Causal world models enhance transferability of learned skills.

Interventional data collection accelerates incremental knowledge building.

Abstract

Despite the recent progress in deep learning and reinforcement learning, transfer and generalization of skills learned on specific tasks is very limited compared to human (or animal) intelligence. The lifelong, incremental building of common sense knowledge might be a necessary component on the way to achieve more general intelligence. A promising direction is to build world models capturing the true physical mechanisms hidden behind the sensorimotor interaction with the environment. Here we explore the idea that inferring the causal structure of the environment could benefit from well-chosen actions as means to collect relevant interventional data.