An AI-powered Bayesian Generative Modeling Approach for Arbitrary Conditional Inference

TL;DR

This paper introduces Bayesian generative modeling (BGM), a flexible framework for arbitrary conditional inference that learns a universal model capable of predicting any conditional distribution without retraining, with strong theoretical guarantees.

Contribution

The paper proposes BGM, a unified Bayesian generative framework that enables arbitrary conditional inference with convergence guarantees and superior predictive performance.

Findings

BGM achieves superior predictive accuracy.

BGM provides principled uncertainty quantification.

Theoretical guarantees for convergence and consistency.

Abstract

Modern data analysis increasingly requires flexible conditional inference P(X_B | X_A) where (X_A, X_B) is an arbitrary partition of observed variable X. Existing approaches are either restricted to a fixed conditioning structure or depend strongly on the distribution of conditioning masks during training. To address these limitations, we introduce Bayesian generative modeling (BGM), a unified framework for arbitrary conditional inference. BGM learns a generative model of X via a stochastic iterative Bayesian updating algorithm in which model parameters and latent variables are updated until convergence. Once trained, any conditional distribution can be obtained without retraining. Empirically, BGM achieves superior predictive performance with posterior predictive intervals, demonstrating that a single learned model can serve as a universal engine for conditional prediction with principled uncertainty quantification. We provide theoretical guarantees for convergence of the stochastic iterative algorithm, statistical consistency, and conditional risk bounds. The proposed BGM framework leverages modern AI to capture complex relationships among variables while adhering to Bayesian principles, offering a promising approach for a wide range of applications in modern data science. Code for BGM is available at https://github.com/liuq-lab/bayesgm. Document of BGM is available at https://bayesgm.readthedocs.io.