Learning to Guide Local Search for MPE Inference in Probabilistic Graphical Models

TL;DR

This paper introduces a neural amortization approach that enhances local search algorithms for MPE inference in PGMs, enabling more effective repeated-query inference by learning to guide move selection.

Contribution

It proposes an attention-based neural framework that predicts move quality, improving local search efficiency and convergence in repeated MPE inference tasks within fixed graphical models.

Findings

Consistent improvement over SLS and GLS+ algorithms.

Effective in high-treewidth benchmark problems.

Seamless integration with existing local search methods.

Abstract

Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs) is a fundamental yet computationally challenging problem arising in domains such as diagnosis, planning, and structured prediction. In many practical settings, the graphical model remains fixed while inference must be performed repeatedly for varying evidence patterns. Stochastic Local Search (SLS) algorithms scale to large models but rely on myopic best-improvement rule that prioritizes immediate likelihood gains and often stagnate in poor local optima. Heuristics such as Guided Local Search (GLS+) partially alleviate this limitation by modifying the search landscape, but their guidance cannot be reused effectively across multiple inference queries on the same model. We propose a neural amortization framework for improving local search in this repeated-query regime. Exploiting the fixed graph structure, we train an attention-based network to score local moves by predicting their ability to reduce Hamming distance to a near-optimal solution. Our approach integrates seamlessly with existing local search procedures, using this signal to balance short-term likelihood gains with long-term promise during neighbor selection. We provide theoretical intuition linking distance-reducing move selection to improved convergence behavior, and empirically demonstrate consistent improvements over SLS and GLS+ on challenging high-treewidth benchmarks in the amortized inference setting.