High-dimensional consistency in score-based and hybrid structure learning

TL;DR

This paper proves the high-dimensional consistency of hybrid score-based methods like GES and ARGES for learning Bayesian networks, demonstrating their scalability and superior performance over constraint-based methods like PC.

Contribution

It establishes the first high-dimensional consistency results for score-based and hybrid methods, introducing adaptive restrictions to ensure consistency.

Findings

ARGES scales to thousands of variables in sparse graphs

Both GES and ARGES outperform the PC algorithm in simulations

Consistency is achieved in classical and high-dimensional settings

Abstract

Main approaches for learning Bayesian networks can be classified as constraint-based, score-based or hybrid methods. Although high-dimensional consistency results are available for constraint-based methods like the PC algorithm, such results have not been proved for score-based or hybrid methods, and most of the hybrid methods have not even shown to be consistent in the classical setting where the number of variables remains fixed and the sample size tends to infinity. In this paper, we show that consistency of hybrid methods based on greedy equivalence search (GES) can be achieved in the classical setting with adaptive restrictions on the search space that depend on the current state of the algorithm. Moreover, we prove consistency of GES and adaptively restricted GES (ARGES) in several sparse high-dimensional settings. ARGES scales well to sparse graphs with thousands of variables and our simulation study indicates that both GES and ARGES generally outperform the PC algorithm.