Probably Approximately Correct Maximum A Posteriori Inference

TL;DR

This paper introduces PAC algorithms for MAP inference that offer provable guarantees within fixed computational budgets, utilizing probabilistic circuits and information-theoretic measures, and demonstrates their effectiveness on various benchmarks.

Contribution

It presents the first PAC algorithms for MAP inference with provable guarantees, leveraging probabilistic circuits and information theory to characterize tractability.

Findings

PAC-MAP algorithms provide optimal solutions with bounded resources.

Probabilistic circuits enable efficient implementation of PAC-MAP.

Experiments show improved performance and guarantees over heuristics.

Abstract

Computing the conditional mode of a distribution, better known as the $\mathit{maximum\ a\ posteriori}$ (MAP) assignment, is a fundamental task in probabilistic inference. However, MAP estimation is generally intractable, and remains hard even under many common structural constraints and approximation schemes. We introduce $\mathit{probably\ approximately\ correct}$ (PAC) algorithms for MAP inference that provide provably optimal solutions under variable and fixed computational budgets. We characterize tractability conditions for PAC-MAP using information theoretic measures that can be estimated from finite samples. Our PAC-MAP solvers are efficiently implemented using probabilistic circuits with appropriate architectures. The randomization strategies we develop can be used either as standalone MAP inference techniques or to improve on popular heuristics, fortifying their solutions with rigorous guarantees. Experiments confirm the benefits of our method in a range of benchmarks.