Approximate Knowledge Compilation by Online Collapsed Importance Sampling

TL;DR

This paper presents collapsed compilation, an innovative approximate inference method combining online variable collapsing with knowledge compilation, effectively exploiting local structure and context-specific independence to improve sampling efficiency in probabilistic graphical models.

Contribution

It introduces a novel online collapsing sampling algorithm that integrates knowledge compilation, enhancing approximate inference by leveraging local structure and context-specific independence.

Findings

Performs well on standard benchmarks

Competitive with state-of-the-art methods under limited exact inference

Outperforms existing methods on several benchmarks

Abstract

We introduce collapsed compilation, a novel approximate inference algorithm for discrete probabilistic graphical models. It is a collapsed sampling algorithm that incrementally selects which variable to sample next based on the partial sample obtained so far. This online collapsing, together with knowledge compilation inference on the remaining variables, naturally exploits local structure and context- specific independence in the distribution. These properties are naturally exploited in exact inference, but are difficult to harness for approximate inference. More- over, by having a partially compiled circuit available during sampling, collapsed compilation has access to a highly effective proposal distribution for importance sampling. Our experimental evaluation shows that collapsed compilation performs well on standard benchmarks. In particular, when the amount of exact inference is equally limited, collapsed compilation is competitive with the state of the art, and outperforms it on several benchmarks.