CREPO: An Open Repository to Benchmark Credal Network Algorithms

TL;DR

CREPO is an open repository of synthetic credal networks with exact inference results, facilitating benchmarking and comparison of algorithms, complemented by a Python tool for easy interaction and validation.

Contribution

It introduces CREPO, a comprehensive benchmark repository with tools for evaluating credal network inference algorithms, including a new heuristic for variable elimination.

Findings

CREPO enables effective benchmarking of credal network algorithms.

The proposed heuristic improves variable elimination efficiency.

Validation shows competitive performance of approximate methods.

Abstract

Credal networks are a popular class of imprecise probabilistic graphical models obtained as a Bayesian network generalization based on, so-called credal, sets of probability mass functions. A Java library called CREMA has been recently released to model, process and query credal networks. Despite the NP-hardness of the (exact) task, a number of algorithms is available to approximate credal network inferences. In this paper we present CREPO, an open repository of synthetic credal networks, provided together with the exact results of inference tasks on these models. A Python tool is also delivered to load these data and interact with CREMA, thus making extremely easy to evaluate and compare existing and novel inference algorithms. To demonstrate such benchmarking scheme, we propose an approximate heuristic to be used inside variable elimination schemes to keep a bound on the maximum number of vertices generated during the combination step. A CREPO-based validation against approximate procedures based on linearization and exact techniques performed in CREMA is finally discussed.