Learning Identifiable Structures Helps Avoid Bias in DNN-based Supervised Causal Learning

TL;DR

This paper introduces SiCL, a novel DNN-based supervised causal learning method that predicts causal structures without bias, outperforming existing approaches by leveraging identifiable structures and a specialized pairwise encoder.

Contribution

The paper proposes SiCL, a new architecture that predicts skeletons and v-structures directly, avoiding bias in traditional node-edge models and enabling consistent causal discovery.

Findings

SiCL outperforms existing DNN-based methods on synthetic benchmarks.

It effectively predicts causal skeletons and v-structures, aligning with MEC theory.

Experimental results demonstrate improved accuracy and consistency.

Abstract

Causal discovery is a structured prediction task that aims to predict causal relations among variables based on their data samples. Supervised Causal Learning (SCL) is an emerging paradigm in this field. Existing Deep Neural Network (DNN)-based methods commonly adopt the "Node-Edge approach", in which the model first computes an embedding vector for each variable-node, then uses these variable-wise representations to concurrently and independently predict for each directed causal-edge. In this paper, we first show that this architecture has some systematic bias that cannot be mitigated regardless of model size and data size. We then propose SiCL, a DNN-based SCL method that predicts a skeleton matrix together with a v-tensor (a third-order tensor representing the v-structures). According to the Markov Equivalence Class (MEC) theory, both the skeleton and the v-structures are identifiable causal structures under the canonical MEC setting, so predictions about skeleton and v-structures do not suffer from the identifiability limit in causal discovery, thus SiCL can avoid the systematic bias in Node-Edge architecture, and enable consistent estimators for causal discovery. Moreover, SiCL is also equipped with a specially designed pairwise encoder module with a unidirectional attention layer to model both internal and external relationships of pairs of nodes. Experimental results on both synthetic and real-world benchmarks show that SiCL significantly outperforms other DNN-based SCL approaches.