Learning Bayesian Networks through Birkhoff Polytope: A Relaxation Method

TL;DR

This paper introduces a relaxation-based framework for learning Bayesian networks by estimating variable orderings and DAG structures efficiently, avoiding NP-hard problems and verifying acyclicity.

Contribution

It proposes a novel relaxation technique for permutation matrix estimation and a decoupled coordinate descent algorithm for sparse Cholesky factor estimation, improving efficiency and accuracy.

Findings

The method accurately recovers DAG structures in simulations.

It converges numerically and is consistent when the variable order is known.

Applied to macro-economic data, it demonstrates practical effectiveness.

Abstract

We establish a novel framework for learning a directed acyclic graph (DAG) when data are generated from a Gaussian, linear structural equation model. It consists of two parts: (1) introduce a permutation matrix as a new parameter within a regularized Gaussian log-likelihood to represent variable ordering; and (2) given the ordering, estimate the DAG structure through sparse Cholesky factor of the inverse covariance matrix. For permutation matrix estimation, we propose a relaxation technique that avoids the NP-hard combinatorial problem of order estimation. Given an ordering, a sparse Cholesky factor is estimated using a cyclic coordinatewise descent algorithm which decouples row-wise. Our framework recovers DAGs without the need for an expensive verification of the acyclicity constraint or enumeration of possible parent sets. We establish numerical convergence of the algorithm, and consistency of the Cholesky factor estimator when the order of variables is known. Through several simulated and macro-economic datasets, we study the scope and performance of the proposed methodology.