Supercharging Bayesian Inference with Reliable AI-Informed Priors

TL;DR

This paper introduces a framework for AI-informed prior elicitation that reduces bias and improves reliability in Bayesian inference by rectifying synthetic data used for prior construction.

Contribution

It proposes a rectification method for AI-informed priors that mitigates model error propagation and enhances inference accuracy and reliability.

Findings

Rectified AI priors significantly reduce bias compared to standard methods.

The approach improves the coverage of credible intervals.

Application to skin disease classification boosts predictive performance.

Abstract

Modern predictive systems encode beliefs that can act as useful prior information for statistical inference in data-limited settings. Using them for prior construction introduces a tradeoff: an informative prior built from a predictive model can sharpen inference from limited data, but also risks propagating error from the model into the posterior. We propose a framework for AI-informed prior elicitation that mitigates this tension by rectifying the AI-induced law that generates synthetic data before using it to inform a prior. The rectified law can be embedded into synthetic data-driven prior elicitation techniques, including as a base measure in a Dirichlet process (DP) prior on the data-generating process. We refer to the resulting prior and corresponding posterior as the rectified AI prior and rectified AI posterior. We establish Gaussian asymptotics for the rectified AI posterior under non-vanishing prior strength and derive a first-order expression for its centering bias. Our rectified AI priors substantially reduce bias compared to standard approaches, improve the coverage of credible intervals, and make AI-powered prior information more reliable. We additionally apply the rectified AI prior to a real skin disease classification task and show that it can meaningfully boost predictive performance.