Investigating the performance of multi-objective optimization when learning Bayesian Networks

TL;DR

This paper explores the use of multi-objective optimization with NSGA-II to improve Bayesian Network structure learning, balancing likelihood and network complexity, and analyzes the quality of solutions on simulated data.

Contribution

It introduces a multi-objective approach to Bayesian Network learning using NSGA-II, explicitly balancing likelihood and network sparsity, and evaluates its effectiveness.

Findings

NSGA-II can find solutions with higher likelihood and fewer arcs.

Solutions often have lower similarity to the true network despite better objective scores.

The approach offers a different trade-off compared to classic methods.

Abstract

Bayesian Networks have been widely used in the last decades in many fields, to describe statistical dependencies among random variables. In general, learning the structure of such models is a problem with considerable theoretical interest that poses many challenges. On the one hand, it is a well-known NP-complete problem, practically hardened by the huge search space of possible solutions. On the other hand, the phenomenon of I-equivalence, i.e., different graphical structures underpinning the same set of statistical dependencies, may lead to multimodal fitness landscapes further hindering maximum likelihood approaches to solve the task. In particular, we exploit the NSGA-II multi-objective optimization procedure in order to explicitly account for both the likelihood of a solution and the number of selected arcs, by setting these as the two objective functions of the method. The aim of this work is to investigate the behavior of NSGA-II and analyse the quality of its solutions. We thus thoroughly examined the optimization results obtained on a wide set of simulated data, by considering both the goodness of the inferred solutions in terms of the objective functions values achieved, and by comparing the retrieved structures with the ground truth, i.e., the networks used to generate the target data. Our results show that NSGA-II can converge to solutions characterized by better likelihood and less arcs than classic approaches, although paradoxically characterized in many cases by a lower similarity with the target network.