Belief propagation for networks with loops: The neighborhoods-intersections-based method

TL;DR

This paper introduces the NIB-method, a new generalized belief propagation algorithm that efficiently accounts for short loops in networks, offering exact solutions in such cases and providing a flexible trade-off between accuracy and computational complexity.

Contribution

The NIB-method improves upon the KCN-method by reducing computational resources needed for belief propagation in networks with short loops, and it offers a new interpolation framework for networks with long loops.

Findings

NIB-method is exact and optimal for networks with only short loops.

NIB-method reduces time complexity compared to KCN-method.

Good agreement between NIB and KCN methods on artificial networks.

Abstract

In order to diminish the damaging effect of loops on belief propagation (BP), the first explicit version of generalized BP for networks, the KCN-method, was recently introduced. Despite its success, the KCN-method spends computational resources inefficiently. Such inefficiencies can quickly turn the exact application of the method unfeasible, since its time complexity increases exponentially with them. This affects for instance tree networks, for which, despite not offering any accuracy advantage with respect to BP, the time complexity of the KCN-method grows exponentially with the nodes' degree. To avoid these issues, we introduce here a new generalized BP scheme, the NIB-method, which only spends computational resources provided they are needed in order to account for correlations in the network. In fact, we show that, given a network with only short loops, the NIB-method is exact and optimal, and we characterize its time complexity reduction with respect to the KCN-method. If long loops are also present, both methods become approximate. In this scenario, we discuss the relation between the methods and we show how to interpolate between them, obtaining a richer family of generalized BP algorithms that trade accuracy for complexity. Lastly, we find a good agreement between the (approximate) KCN and NIB methods when computing the partition function for two artificial networks.