Efficient Differentiable Causal Discovery via Reliable Super-Structure Learning

TL;DR

This paper introduces ALVGL, a method that enhances differentiable causal discovery by learning super-structures through sparse and low-rank decomposition, improving accuracy and efficiency especially in high-dimensional and confounded data.

Contribution

ALVGL is a novel approach that efficiently learns super-structures to guide causal discovery, applicable across various models and settings, with proven state-of-the-art results.

Findings

Achieves state-of-the-art accuracy on synthetic and real datasets.

Significantly improves optimization efficiency in causal discovery.

Effective across Gaussian, non-Gaussian, and confounded models.

Abstract

Recently, differentiable causal discovery has emerged as a promising approach to improve the accuracy and efficiency of existing methods. However, when applied to high-dimensional data or data with latent confounders, these methods, often based on off-the-shelf continuous optimization algorithms, struggle with the vast search space, the complexity of the objective function, and the nontrivial nature of graph-theoretical constraints. As a result, there has been a surge of interest in leveraging super-structures to guide the optimization process. Nonetheless, learning an appropriate super-structure at the right level of granularity, and doing so efficiently across various settings, presents significant challenges. In this paper, we propose ALVGL, a novel and general enhancement to the differentiable causal discovery pipeline. ALVGL employs a sparse and low-rank decomposition to learn the precision matrix of the data. We design an ADMM procedure to optimize this decomposition, identifying components in the precision matrix that are most relevant to the underlying causal structure. These components are then combined to construct a super-structure that is provably a superset of the true causal graph. This super-structure is used to initialize a standard differentiable causal discovery method with a more focused search space, thereby improving both optimization efficiency and accuracy. We demonstrate the versatility of ALVGL by instantiating it across a range of structural causal models, including both Gaussian and non-Gaussian settings, with and without unmeasured confounders. Extensive experiments on synthetic and real-world datasets show that ALVGL not only achieves state-of-the-art accuracy but also significantly improves optimization efficiency, making it a reliable and effective solution for differentiable causal discovery.