IBIA: Bayesian Inference via Incremental Build-Infer-Approximate operations on Clique Trees

TL;DR

IBIA introduces an incremental approach to approximate inference in Bayesian networks by constructing linked clique tree forests with bounded clique sizes, improving accuracy and efficiency over existing methods.

Contribution

The paper presents a novel incremental build-infer-approximate paradigm that constructs linked clique tree forests with controlled clique sizes for better approximate inference.

Findings

Significant reduction in inference error compared to other methods.

Efficient approximate inference of partition functions and marginals.

Validated on over 500 benchmarks with competitive runtimes.

Abstract

Exact inference in Bayesian networks is intractable and has an exponential dependence on the size of the largest clique in the corresponding clique tree (CT), necessitating approximations. Factor based methods to bound clique sizes are more accurate than structure based methods, but expensive since they involve inference of beliefs in a large number of candidate structure or region graphs. We propose an alternative approach for approximate inference based on an incremental build-infer-approximate (IBIA) paradigm, which converts the Bayesian network into a data structure containing a sequence of linked clique tree forests (SLCTF), with clique sizes bounded by a user-specified value. In the incremental build stage of this approach, CTFs are constructed incrementally by adding variables to the CTFs as long as clique sizes are within the specified bound. Once the clique size constraint is reached, the CTs in the CTF are calibrated in the infer stage of IBIA. The resulting clique beliefs are used in the approximate phase to get an approximate CTF with reduced clique sizes. The approximate CTF forms the starting point for the next CTF in the sequence. These steps are repeated until all variables are added to a CTF in the sequence. We prove that our algorithm for incremental construction of clique trees always generates a valid CT and our approximation technique preserves the joint beliefs of the variables within a clique. Based on this, we show that the SLCTF data structure can be used for efficient approximate inference of partition function and prior and posterior marginals. More than 500 benchmarks were used to test the method and the results show a significant reduction in error when compared to other approximate methods, with competitive runtimes.