Graph-attention-based Casual Discovery with Trust Region-navigated Clipping Policy Optimization

TL;DR

This paper introduces a novel reinforcement learning approach with trust region navigation and a refined graph attention encoder for more robust and efficient causal structure discovery, outperforming previous methods on synthetic and benchmark datasets.

Contribution

It proposes a trust region-navigated clipping policy optimization and a new SDGAT encoder to enhance causal discovery performance and stability.

Findings

Outperforms previous RL methods in synthetic datasets.

Achieves better robustness and efficiency in causal structure learning.

Demonstrates superior results on benchmark datasets.

Abstract

In many domains of empirical sciences, discovering the causal structure within variables remains an indispensable task. Recently, to tackle with unoriented edges or latent assumptions violation suffered by conventional methods, researchers formulated a reinforcement learning (RL) procedure for causal discovery, and equipped REINFORCE algorithm to search for the best-rewarded directed acyclic graph. The two keys to the overall performance of the procedure are the robustness of RL methods and the efficient encoding of variables. However, on the one hand, REINFORCE is prone to local convergence and unstable performance during training. Neither trust region policy optimization, being computationally-expensive, nor proximal policy optimization (PPO), suffering from aggregate constraint deviation, is decent alternative for combinatory optimization problems with considerable individual subactions. We propose a trust region-navigated clipping policy optimization method for causal discovery that guarantees both better search efficiency and steadiness in policy optimization, in comparison with REINFORCE, PPO and our prioritized sampling-guided REINFORCE implementation. On the other hand, to boost the efficient encoding of variables, we propose a refined graph attention encoder called SDGAT that can grasp more feature information without priori neighbourhood information. With these improvements, the proposed method outperforms former RL method in both synthetic and benchmark datasets in terms of output results and optimization robustness.