Concept-free Causal Disentanglement with Variational Graph Auto-Encoder

TL;DR

This paper introduces a novel unsupervised method for causal disentanglement in graph data, using a variational graph auto-encoder that models causal structures without relying on concept labels.

Contribution

It proposes a concept-free causal disentanglement framework with a new variational graph auto-encoder, improving causal structure learning from graph data.

Findings

Achieves up to 29% AUC improvement over baselines.

Demonstrates the effectiveness of concept-free causal disentanglement.

Enhances meta-learning with causal structure modeling.

Abstract

In disentangled representation learning, the goal is to achieve a compact representation that consists of all interpretable generative factors in the observational data. Learning disentangled representations for graphs becomes increasingly important as graph data rapidly grows. Existing approaches often rely on Variational Auto-Encoder (VAE) or its causal structure learning-based refinement, which suffer from sub-optimality in VAEs due to the independence factor assumption and unavailability of concept labels, respectively. In this paper, we propose an unsupervised solution, dubbed concept-free causal disentanglement, built on a theoretically provable tight upper bound approximating the optimal factor. This results in an SCM-like causal structure modeling that directly learns concept structures from data. Based on this idea, we propose Concept-free Causal VGAE (CCVGAE) by incorporating a novel causal disentanglement layer into Variational Graph Auto-Encoder. Furthermore, we prove concept consistency under our concept-free causal disentanglement framework, hence employing it to enhance the meta-learning framework, called concept-free causal Meta-Graph (CC-Meta-Graph). We conduct extensive experiments to demonstrate the superiority of the proposed models: CCVGAE and CC-Meta-Graph, reaching up to $29\%$ and $11\%$ absolute improvements over baselines in terms of AUC, respectively.