Free Energy Manifold: Score-Based Inference for Hybrid Bayesian Networks

TL;DR

The paper introduces the Free Energy Manifold (FEM), a score-trained energy model for hybrid Bayesian networks, addressing inference challenges and mode-bridge artifacts with valley regularization.

Contribution

FEM is a novel energy-based approach for hybrid Bayesian networks that improves inference accuracy and calibration, especially in multimodal and compositional scenarios.

Findings

FEM significantly reduces KL divergence compared to classical baselines.

Valley regularization restores near-uniform posteriors in mode-bridge regions.

FEM performs well in high-cardinality discrete-parent and multi-leaf evidence settings.

Abstract

We introduce the Free Energy Manifold (FEM), a score-trained conditional energy model specialized for inference in hybrid Bayesian networks with discrete and continuous variables. FEM represents each conditional factor as an energy landscape over learned discrete-parent embeddings and continuous observations, enabling posterior evaluation, generative sampling, and compositional inference across multiple continuous leaves by energy addition under conditional independence. A central finding is the mode-bridge artifact: standard conditional energy models can create low-energy ridges between separated modes of the same class, producing overconfident posteriors at off-data interior points. We analyze this failure and propose valley regularization, an off-data calibration term that restores near-uniform posteriors in such regions while preserving in-data fit. Across synthetic multimodal hybrid-BN benchmarks, FEM substantially reduces KL divergence relative to classical baselines and a vanilla conditional EBM, including large gains at mode-bridge midpoint queries and in multi-leaf evidence composition. We also evaluate high-cardinality discrete-parent settings and a UCI Breast Cancer sanity check, showing that FEM is most useful when multimodal or compositional Bayesian-network inference is required, while discriminative classifiers remain preferable for closed-world classification tasks.