MARLIN: Multi-Agent Reinforcement Learning for Incremental DAG Discovery

TL;DR

MARLIN introduces a multi-agent reinforcement learning approach for efficient incremental discovery of causal DAG structures from data, improving speed and accuracy over existing methods.

Contribution

The paper presents MARLIN, a novel multi-agent RL framework that enhances DAG learning efficiency and effectiveness through a factored action space and incremental updating.

Findings

Outperforms state-of-the-art methods in efficiency.

Achieves higher accuracy in DAG discovery.

Effective on both synthetic and real datasets.

Abstract

Uncovering causal structures from observational data is crucial for understanding complex systems and making informed decisions. While reinforcement learning (RL) has shown promise in identifying these structures in the form of a directed acyclic graph (DAG), existing methods often lack efficiency, making them unsuitable for online applications. In this paper, we propose MARLIN, an efficient multi agent RL based approach for incremental DAG learning. MARLIN uses a DAG generation policy that maps a continuous real valued space to the DAG space as an intra batch strategy, then incorporates two RL agents state specific and state invariant to uncover causal relationships and integrates these agents into an incremental learning framework. Furthermore, the framework leverages a factored action space to enhance parallelization efficiency. Extensive experiments on synthetic and real datasets demonstrate that MARLIN outperforms state of the art methods in terms of both efficiency and effectiveness.