An Asymptotically Optimal Coordinate Descent Algorithm for Learning Bayesian Networks from Gaussian Models

TL;DR

This paper introduces a new coordinate descent algorithm for learning Bayesian networks from Gaussian data, offering convergence, optimality, and statistical guarantees, and demonstrating scalability and effectiveness through experiments.

Contribution

A novel coordinate descent method with proven convergence, optimality, and statistical guarantees for learning Bayesian networks from Gaussian models.

Findings

Algorithm converges to a coordinate-wise minimum.

Objective value approaches the optimal as sample size increases.

Method is scalable and performs well on synthetic and real data.

Abstract

This paper studies the problem of learning Bayesian networks from continuous observational data, generated according to a linear Gaussian structural equation model. We consider an $\ell_0$-penalized maximum likelihood estimator for this problem which is known to have favorable statistical properties but is computationally challenging to solve, especially for medium-sized Bayesian networks. We propose a new coordinate descent algorithm to approximate this estimator and prove several remarkable properties of our procedure: the algorithm converges to a coordinate-wise minimum, and despite the non-convexity of the loss function, as the sample size tends to infinity, the objective value of the coordinate descent solution converges to the optimal objective value of the $\ell_0$-penalized maximum likelihood estimator. Finite-sample statistical consistency guarantees are also established. To the best of our knowledge, our proposal is the first coordinate descent procedure endowed with optimality and statistical guarantees in the context of learning Bayesian networks. Numerical experiments on synthetic and real data demonstrate that our coordinate descent method can obtain near-optimal solutions while being scalable.