Strengthening Probabilistic Graphical Models: The Purge-and-merge Algorithm

TL;DR

The paper introduces the purge-and-merge algorithm, which iteratively simplifies loopy probabilistic graphical models into tree structures to enable efficient and exact inference, outperforming existing methods on certain constraint satisfaction problems.

Contribution

The paper presents a novel purge-and-merge algorithm that reduces complex loopy PGMs to tree structures without exponential blow-up, improving inference efficiency and accuracy.

Findings

Outperformed existing PGM approaches on Sudoku, Fill-a-pix, and Kakuro tasks.

Effectively simplifies loopy PGMs into tree structures.

Demonstrated potential for extension to general PGM inference.

Abstract

Probabilistic graphical models (PGMs) are powerful tools for solving systems of complex relationships over a variety of probability distributions. However, while tree-structured PGMs always result in efficient and exact solutions, inference on graph (or loopy) structured PGMs is not guaranteed to discover the optimal solutions. It is in principle possible to convert loopy PGMs to an equivalent tree structure, but this is usually impractical for interesting problems due to exponential blow-up. To address this, we developed the purge-and-merge algorithm. This algorithm iteratively nudges a malleable graph structure towards a tree structure by selectively merging factors. The merging process is designed to avoid exponential blow-up by way of sparse structures from which redundancy is purged as the algorithm progresses. We set up tasks to test the algorithm on constraint-satisfaction puzzles such as Sudoku, Fill-a-pix, and Kakuro, and it outperformed other PGM-based approaches reported in the literature. While the tasks we set focussed on the binary logic of CSP, we believe the purge-and-merge algorithm could be extended to general PGM inference.