Generalization of graph network inferences in higher-order graphical models

TL;DR

This paper introduces RF-GNN, a neural network approach for fast approximate inference in complex graphical models with higher-order interactions, demonstrating superior performance and generalization over traditional methods like Belief Propagation.

Contribution

The paper presents RF-GNN, a novel recurrent graph neural network designed to improve inference in higher-order graphical models, especially under challenging conditions.

Findings

RF-GNN outperforms Belief Propagation on various graphical models.

RF-GNN generalizes well to different graph sizes and distributions.

RF-GNN is more robust under high noise levels.

Abstract

Probabilistic graphical models provide a powerful tool to describe complex statistical structure, with many real-world applications in science and engineering from controlling robotic arms to understanding neuronal computations. A major challenge for these graphical models is that inferences such as marginalization are intractable for general graphs. These inferences are often approximated by a distributed message-passing algorithm such as Belief Propagation, which does not always perform well on graphs with cycles, nor can it always be easily specified for complex continuous probability distributions. Such difficulties arise frequently in expressive graphical models that include intractable higher-order interactions. In this paper we define the Recurrent Factor Graph Neural Network (RF-GNN) to achieve fast approximate inference on graphical models that involve many-variable interactions. Experimental results on several families of graphical models demonstrate the out-of-distribution generalization capability of our method to different sized graphs, and indicate the domain in which our method outperforms Belief Propagation (BP). Moreover, we test the RF-GNN on a real-world Low-Density Parity-Check dataset as a benchmark along with other baseline models including BP variants and other GNN methods. Overall we find that RF-GNNs outperform other methods under high noise levels.