Learning Bayesian Network Structure from Correlation-Immune Data

TL;DR

This paper introduces an extension to the Sparse Candidate algorithm using a technique called 'skewing' to effectively learn Bayesian network structures involving correlation-immune relationships, which traditional heuristics struggle to identify.

Contribution

The paper presents a novel extension to the Sparse Candidate algorithm that enables learning of correlation-immune relationships in Bayesian networks using the skewing technique.

Findings

Successfully learns correlation-immune relationships with lower computational cost.

Extends the capability of existing algorithms to handle complex relationships.

Demonstrates effectiveness on problems with parity-like dependencies.

Abstract

Searching the complete space of possible Bayesian networks is intractable for problems of interesting size, so Bayesian network structure learning algorithms, such as the commonly used Sparse Candidate algorithm, employ heuristics. However, these heuristics also restrict the types of relationships that can be learned exclusively from data. They are unable to learn relationships that exhibit "correlation-immunity", such as parity. To learn Bayesian networks in the presence of correlation-immune relationships, we extend the Sparse Candidate algorithm with a technique called "skewing". This technique uses the observation that relationships that are correlation-immune under a specific input distribution may not be correlation-immune under another, sufficiently different distribution. We show that by extending Sparse Candidate with this technique we are able to discover relationships between random variables that are approximately correlation-immune, with a significantly lower computational cost than the alternative of considering multiple parents of a node at a time.