Region-based Energy Neural Network for Approximate Inference

TL;DR

This paper introduces RENN, a neural network model that directly minimizes region-based free energy for efficient inference in Markov random fields, outperforming traditional and neural methods.

Contribution

The paper presents RENN, a novel neural network approach that avoids message passing, works on general MRFs, and improves inference and learning efficiency.

Findings

RENN outperforms mean field, loopy BP, GBP, and neural models in experiments.

RENN is faster than message-passing algorithms due to direct energy minimization.

RENN does not require sampling and can be used for MRF learning.

Abstract

Region-based free energy was originally proposed for generalized belief propagation (GBP) to improve loopy belief propagation (loopy BP). In this paper, we propose a neural network based energy model for inference in general Markov random fields (MRFs), which directly minimizes the region-based free energy defined on region graphs. We term our model Region-based Energy Neural Network (RENN). Unlike message-passing algorithms, RENN avoids iterative message propagation and is faster. Also different from recent deep neural network based models, inference by RENN does not require sampling, and RENN works on general MRFs. RENN can also be employed for MRF learning. Our experiments on marginal distribution estimation, partition function estimation, and learning of MRFs show that RENN outperforms the mean field method, loopy BP, GBP, and the state-of-the-art neural network based model.