On Local Optima in Learning Bayesian Networks

TL;DR

This paper introduces the KES algorithm for learning Bayesian networks, which balances greediness and randomness to explore multiple local optima, outperforming traditional methods in finding better solutions.

Contribution

The paper presents KES, a novel algorithm that generalizes GES by allowing a trade-off between greediness and randomness, with theoretical guarantees and empirical validation.

Findings

KES often finds better local optima than GES

The number of local optima in Bayesian network learning is large

KES can asymptotically return any inclusion optimal BN

Abstract

This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima. When greediness is set at maximum, KES corresponds to the greedy equivalence search algorithm (GES). When greediness is kept at minimum, we prove that under mild assumptions KES asymptotically returns any inclusion optimal BN with nonzero probability. Experimental results for both synthetic and real data are reported showing that KES often finds a better local optima than GES. Moreover, we use KES to experimentally confirm that the number of different local optima is often huge.