Towards intervention-centric causal reasoning in learning agents

TL;DR

This paper proposes a meta-learning approach enabling agents to learn intervention-centric causal reasoning in high-dimensional environments without predefined intervention actions, leveraging deep reinforcement learning to discover manipulations and causal structures.

Contribution

It introduces a meta-reinforcement learning framework that allows agents to learn targeted interventions and causal representations from observations in complex environments.

Findings

Meta-learning enables causal reasoning without explicit intervention actions.

Agents learn to manipulate environments and infer causal structures.

Approach transfers knowledge across observational causal tasks.

Abstract

Interventions are central to causal learning and reasoning. Yet ultimately an intervention is an abstraction: an agent embedded in a physical environment (perhaps modeled as a Markov decision process) does not typically come equipped with the notion of an intervention -- its action space is typically ego-centric, without actions of the form `intervene on X'. Such a correspondence between ego-centric actions and interventions would be challenging to hard-code. It would instead be better if an agent learnt which sequence of actions allow it to make targeted manipulations of the environment, and learnt corresponding representations that permitted learning from observation. Here we show how a meta-learning approach can be used to perform causal learning in this challenging setting, where the action-space is not a set of interventions and the observation space is a high-dimensional space with a latent causal structure. A meta-reinforcement learning algorithm is used to learn relationships that transfer on observational causal learning tasks. This work shows how advances in deep reinforcement learning and meta-learning can provide intervention-centric causal learning in high-dimensional environments with a latent causal structure.