SymCircuit: Bayesian Structure Inference for Tractable Probabilistic Circuits via Entropy-Regularized Reinforcement Learning

TL;DR

SymCircuit introduces a reinforcement learning approach with entropy regularization for probabilistic circuit structure inference, improving efficiency and accuracy over greedy algorithms, and ensuring valid circuit generation at each step.

Contribution

It replaces greedy search with a learned generative policy trained via entropy-regularized reinforcement learning, incorporating a novel grammar-constrained Transformer for probabilistic circuit learning.

Findings

Achieves >10x sample efficiency on NLTCS dataset.

Closes 93% of the gap to LearnSPN on NLTCS.

Demonstrates scalability on a 69-variable Plants dataset.

Abstract

Probabilistic circuit (PC) structure learning is hampered by greedy algorithms that make irreversible, locally optimal decisions. We propose SymCircuit, which replaces greedy search with a learned generative policy trained via entropy-regularized reinforcement learning. Instantiating the RL-as-inference framework in the PC domain, we show the optimal policy is a tempered Bayesian posterior, recovering the exact posterior when the regularization temperature is set inversely proportional to the dataset size. The policy is implemented as SymFormer, a grammar-constrained autoregressive Transformer with tree-relative self-attention that guarantees valid circuits at every generation step. We introduce option-level REINFORCE, restricting gradient updates to structural decisions rather than all tokens, yielding an SNR (signal to noise ratio) improvement and >10 times sample efficiency gain on the NLTCS dataset. A three-layer uncertainty decomposition (structural via model averaging, parametric via the delta method, leaf via conjugate Dirichlet-Categorical propagation) is grounded in the multilinear polynomial structure of PC outputs. On NLTCS, SymCircuit closes 93% of the gap to LearnSPN; preliminary results on Plants (69 variables) suggest scalability.