# Optimizing regularized Cholesky score for order-based learning of Bayesian networks

## TL;DR

This paper introduces ARCS, a novel method for learning Bayesian network structures by optimizing a regularized Cholesky score through annealing and efficient algorithms, improving accuracy over existing methods.

## Contribution

The paper presents a new structure learning algorithm for Bayesian networks that combines annealing with a fast proximal gradient method, avoiding acyclicity checks and enhancing search efficiency.

## Key findings

- ARCS outperforms existing methods in numerical experiments.
- The approach effectively searches over DAGs without explicit acyclicity verification.
- It improves the accuracy of learned Bayesian networks from observational and experimental data.

## Abstract

Bayesian networks are a class of popular graphical models that encode causal and conditional independence relations among variables by directed acyclic graphs (DAGs). We propose a novel structure learning method, annealing on regularized Cholesky score (ARCS), to search over topological sorts, or permutations of nodes, for a high-scoring Bayesian network. Our scoring function is derived from regularizing Gaussian DAG likelihood, and its optimization gives an alternative formulation of the sparse Cholesky factorization problem from a statistical viewpoint, which is of independent interest. We combine global simulated annealing over permutations with a fast proximal gradient algorithm, operating on triangular matrices of edge coefficients, to compute the score of any permutation. Combined, the two approaches allow us to quickly and effectively search over the space of DAGs without the need to verify the acyclicity constraint or to enumerate possible parent sets given a candidate topological sort. The annealing aspect of the optimization is able to consistently improve the accuracy of DAGs learned by local search algorithms. In addition, we develop several techniques to facilitate the structure learning, including pre-annealing data-driven tuning parameter selection and post-annealing constraint-based structure refinement. Through extensive numerical comparisons, we show that ARCS achieves substantial improvements over existing methods, demonstrating its great potential to learn Bayesian networks from both observational and experimental data.