The Limits of Tractable Marginalization

TL;DR

This paper investigates the computational limits of marginalization, showing that some functions with tractable marginalization cannot be efficiently represented by known circuit models, under standard complexity assumptions.

Contribution

It demonstrates that not all functions with polynomial-time marginalization can be succinctly expressed by known polynomial-size arithmetic circuits, revealing fundamental limitations.

Findings

Certain simple functions with tractable marginalization lack efficient circuit representations.

Hierarchy of complexity classes for stronger marginalization forms is identified.

Efficient real RAM algorithms imply small circuits for multilinear representations.

Abstract

Marginalization -- summing a function over all assignments to a subset of its inputs -- is a fundamental computational problem with applications from probabilistic inference to formal verification. Despite its computational hardness in general, there exist many classes of functions (e.g., probabilistic models) for which marginalization remains tractable, and they can be commonly expressed by polynomial size arithmetic circuits computing multilinear polynomials. This raises the question, can all functions with polynomial time marginalization algorithms be succinctly expressed by such circuits? We give a negative answer, exhibiting simple functions with tractable marginalization yet no efficient representation by known models, assuming $\textsf{FP}\neq\#\textsf{P}$ (an assumption implied by $\textsf{P} \neq \textsf{NP}$). To this end, we identify a hierarchy of complexity classes corresponding to stronger forms of marginalization, all of which are efficiently computable on the known circuit models. We conclude with a completeness result, showing that whenever there is an efficient real RAM performing virtual evidence marginalization for a function, then there are small circuits for that function's multilinear representation.