Markov Equivalence and Consistency in Differentiable Structure Learning

TL;DR

This paper investigates differentiable structure learning of DAGs, demonstrating how regularization can identify the sparsest model within the Markov equivalence class without strong identifiability assumptions.

Contribution

It introduces a regularization approach that ensures the identification of the sparsest DAG in the equivalence class, even without identifiable parametrizations, and generalizes results beyond Gaussian models.

Findings

Proper regularization defines a score for sparsity

The method recovers the Markov equivalence class under faithfulness

Empirical validation shows compatibility with standard optimizers

Abstract

Existing approaches to differentiable structure learning of directed acyclic graphs (DAGs) rely on strong identifiability assumptions in order to guarantee that global minimizers of the acyclicity-constrained optimization problem identifies the true DAG. Moreover, it has been observed empirically that the optimizer may exploit undesirable artifacts in the loss function. We explain and remedy these issues by studying the behavior of differentiable acyclicity-constrained programs under general likelihoods with multiple global minimizers. By carefully regularizing the likelihood, it is possible to identify the sparsest model in the Markov equivalence class, even in the absence of an identifiable parametrization. We first study the Gaussian case in detail, showing how proper regularization of the likelihood defines a score that identifies the sparsest model. Assuming faithfulness, it also recovers the Markov equivalence class. These results are then generalized to general models and likelihoods, where the same claims hold. These theoretical results are validated empirically, showing how this can be done using standard gradient-based optimizers, thus paving the way for differentiable structure learning under general models and losses.