TabMGP: Martingale posterior with TabPFN

TL;DR

TabMGP combines martingale posterior methodology with TabPFN to provide reliable uncertainty quantification in tabular data, outperforming traditional Bayesian methods and handcrafted approaches.

Contribution

This paper introduces TabMGP, a novel approach that integrates martingale posterior with TabPFN for improved uncertainty quantification in tabular data analysis.

Findings

TabMGP achieves near-nominal coverage in credible sets.

It often outperforms handcrafted MGP and Bayesian baselines.

Provides epistemic uncertainty for non-parametric models.

Abstract

Bayesian inference provides principled uncertainty quantification but is often limited by challenges of prior and likelihood elicitation. The martingale posterior (MGP) (Fong et al., 2023) offers an alternative by replacing these requirements with a predictive rule. Additionally MGP focuses inference on parameters defined through a loss function. This framework is especially resonant in the era of foundation transformers; practitioners increasingly leverage models like TabPFN for their state-of-the-art capabilities, yet often require epistemic uncertainty for a scientific estimand $\theta$ that need not parameterise the model's implicit latent model. The MGP provides the mechanism to recover these posterior distributions. We introduce TabMGP, an MGP built on TabPFN for tabular data. TabMGP produces credible sets with near-nominal coverage and often outperforms both handcrafted MGP constructions and standard Bayesian baselines.