Smoothness and Structure Learning by Proxy

TL;DR

This paper demonstrates that using a Gaussian Process proxy for scoring Bayesian network structures can significantly speed up the search process while maintaining or improving accuracy, by bounding the score function's smoothness.

Contribution

It proves the theoretical validity of proxy-based scoring in Bayesian network learning and empirically shows its efficiency and effectiveness.

Findings

Proxy-based search achieves comparable or better scores.

Significant reduction in search time.

Bounded smoothness of the scoring function supports proxy use.

Abstract

As data sets grow in size, the ability of learning methods to find structure in them is increasingly hampered by the time needed to search the large spaces of possibilities and generate a score for each that takes all of the observed data into account. For instance, Bayesian networks, the model chosen in this paper, have a super-exponentially large search space for a fixed number of variables. One possible method to alleviate this problem is to use a proxy, such as a Gaussian Process regressor, in place of the true scoring function, training it on a selection of sampled networks. We prove here that the use of such a proxy is well-founded, as we can bound the smoothness of a commonly-used scoring function for Bayesian network structure learning. We show here that, compared to an identical search strategy using the network?s exact scores, our proxy-based search is able to get equivalent or better scores on a number of data sets in a fraction of the time.