Learning Sparse Fixed-Structure Gaussian Bayesian Networks

TL;DR

This paper investigates algorithms for learning fixed-structure Gaussian Bayesian networks, demonstrating near-optimal sample complexity and robustness to contamination, with practical comparisons of different methods.

Contribution

The paper introduces and analyzes new algorithms, BatchAvgLeastSquares and CauchyEst, that achieve near-optimal sample complexity and robustness for learning Gaussian Bayesian networks.

Findings

LeastSquares has near-optimal sample complexity for uncontaminated data.

CauchyEst and BatchAvgLeastSquares outperform in contaminated or misspecified settings.

CauchyEstTree is effective for polytrees with near-optimal sample complexity.

Abstract

Gaussian Bayesian networks (a.k.a. linear Gaussian structural equation models) are widely used to model causal interactions among continuous variables. In this work, we study the problem of learning a fixed-structure Gaussian Bayesian network up to a bounded error in total variation distance. We analyze the commonly used node-wise least squares regression (LeastSquares) and prove that it has a near-optimal sample complexity. We also study a couple of new algorithms for the problem: - BatchAvgLeastSquares takes the average of several batches of least squares solutions at each node, so that one can interpolate between the batch size and the number of batches. We show that BatchAvgLeastSquares also has near-optimal sample complexity. - CauchyEst takes the median of solutions to several batches of linear systems at each node. We show that the algorithm specialized to polytrees, CauchyEstTree, has near-optimal sample complexity. Experimentally, we show that for uncontaminated, realizable data, the LeastSquares algorithm performs best, but in the presence of contamination or DAG misspecification, CauchyEst/CauchyEstTree and BatchAvgLeastSquares respectively perform better.