Causal Discovery via Bayesian Optimization

TL;DR

This paper introduces DrBO, a Bayesian optimization-based framework for causal discovery that efficiently identifies high-scoring DAGs using neural networks, outperforming existing methods in accuracy and computational efficiency.

Contribution

The paper proposes DrBO, a novel DAG learning method utilizing dropout neural networks within Bayesian optimization to improve scalability and accuracy in causal discovery.

Findings

DrBO finds accurate DAGs with fewer evaluations.

It outperforms state-of-the-art methods in synthetic and real data.

It is computationally more efficient and scalable.

Abstract

Existing score-based methods for directed acyclic graph (DAG) learning from observational data struggle to recover the causal graph accurately and sample-efficiently. To overcome this, in this study, we propose DrBO (DAG recovery via Bayesian Optimization)-a novel DAG learning framework leveraging Bayesian optimization (BO) to find high-scoring DAGs. We show that, by sophisticatedly choosing the promising DAGs to explore, we can find higher-scoring ones much more efficiently. To address the scalability issues of conventional BO in DAG learning, we replace Gaussian Processes commonly employed in BO with dropout neural networks, trained in a continual manner, which allows for (i) flexibly modeling the DAG scores without overfitting, (ii) incorporation of uncertainty into the estimated scores, and (iii) scaling with the number of evaluations. As a result, DrBO is computationally efficient and can find the accurate DAG in fewer trials and less time than existing state-of-the-art methods. This is demonstrated through an extensive set of empirical evaluations on many challenging settings with both synthetic and real data. Our implementation is available at https://github.com/baosws/DrBO.