Exact Maximum Margin Structure Learning of Bayesian Networks

TL;DR

This paper introduces an exact branch-and-bound method within a linear programming framework to learn Bayesian network structures that maximize the probabilistic soft margin, improving discriminative performance.

Contribution

It presents a novel exact optimization approach for discriminative Bayesian network structure learning using soft margin maximization.

Findings

Outperforms generative Bayesian networks in classification tasks

Competes effectively with support vector machines

Provides worst-case sub-optimality bounds during learning

Abstract

Recently, there has been much interest in finding globally optimal Bayesian network structures. These techniques were developed for generative scores and can not be directly extended to discriminative scores, as desired for classification. In this paper, we propose an exact method for finding network structures maximizing the probabilistic soft margin, a successfully applied discriminative score. Our method is based on branch-and-bound techniques within a linear programming framework and maintains an any-time solution, together with worst-case sub-optimality bounds. We apply a set of order constraints for enforcing the network structure to be acyclic, which allows a compact problem representation and the use of general-purpose optimization techniques. In classification experiments, our methods clearly outperform generatively trained network structures and compete with support vector machines.