Causal Induction from Visual Observations for Goal Directed Tasks

TL;DR

This paper introduces a learning-based method for enabling artificial agents to perform causal reasoning from visual observations, allowing them to adapt to new goal-directed tasks in environments with unseen causal structures.

Contribution

It develops an attention-based approach to induce causal graphs from visual data and integrates this into goal-conditional policies for improved task generalization.

Findings

Effective causal graph induction from visual observations.

Successful generalization to new environments with unseen causal structures.

Improved task completion in novel environments.

Abstract

Causal reasoning has been an indispensable capability for humans and other intelligent animals to interact with the physical world. In this work, we propose to endow an artificial agent with the capability of causal reasoning for completing goal-directed tasks. We develop learning-based approaches to inducing causal knowledge in the form of directed acyclic graphs, which can be used to contextualize a learned goal-conditional policy to perform tasks in novel environments with latent causal structures. We leverage attention mechanisms in our causal induction model and goal-conditional policy, enabling us to incrementally generate the causal graph from the agent's visual observations and to selectively use the induced graph for determining actions. Our experiments show that our method effectively generalizes towards completing new tasks in novel environments with previously unseen causal structures.