Renormalized Normalized Maximum Likelihood and Three-Part Code Criteria For Learning Gaussian Networks

TL;DR

This paper introduces two new scoring metrics based on MDL principles for learning Gaussian Bayesian networks directly in the continuous domain, improving convergence and accuracy over traditional methods.

Contribution

The paper proposes the renormalized normalized maximum likelihood and three-part code criteria for continuous Bayesian network learning, which are hyperparameter-free, decomposable, and asymptotically consistent.

Findings

Proposed metrics outperform BIC/AIC in convergence rate.

RNML achieves the fastest convergence to the true network.

Metrics result in smaller structural Hamming distances.

Abstract

Score based learning (SBL) is a promising approach for learning Bayesian networks in the discrete domain. However, when employing SBL in the continuous domain, one is either forced to move the problem to the discrete domain or use metrics such as BIC/AIC, and these approaches are often lacking. Discretization can have an undesired impact on the accuracy of the results, and BIC/AIC can fall short of achieving the desired accuracy. In this paper, we introduce two new scoring metrics for scoring Bayesian networks in the continuous domain: the three-part minimum description length and the renormalized normalized maximum likelihood metric. We rely on the minimum description length principle in formulating these metrics. The metrics proposed are free of hyperparameters, decomposable, and are asymptotically consistent. We evaluate our solution by studying the convergence rate of the learned graph to the generating network and, also, the structural hamming distance of the learned graph to the generating network. Our evaluations show that the proposed metrics outperform their competitors, the BIC/AIC metrics. Furthermore, using the proposed RNML metric, SBL will have the fastest rate of convergence with the smallest structural hamming distance to the generating network.