Structure Mapping for Transferability of Causal Models

TL;DR

This paper introduces a transfer-learning framework that uses object-oriented causal models to transfer knowledge across environments with similar causal dynamics but different perceptual features, enhancing reinforcement learning.

Contribution

It proposes a novel causal structure learning method for transferability in object-based environments, combining causal models with reinforcement learning techniques.

Findings

Effective transfer of causal knowledge in gridworld environments.

Improved reinforcement learning performance using causal structure transfer.

Demonstrated advantages over purely model-free approaches.

Abstract

Human beings learn causal models and constantly use them to transfer knowledge between similar environments. We use this intuition to design a transfer-learning framework using object-oriented representations to learn the causal relationships between objects. A learned causal dynamics model can be used to transfer between variants of an environment with exchangeable perceptual features among objects but with the same underlying causal dynamics. We adapt continuous optimization for structure learning techniques to explicitly learn the cause and effects of the actions in an interactive environment and transfer to the target domain by categorization of the objects based on causal knowledge. We demonstrate the advantages of our approach in a gridworld setting by combining causal model-based approach with model-free approach in reinforcement learning.