Inference with Discriminative Posterior
Jarkko Saloj\"arvi, Kai Puolam\"aki, Eerika Savia, Samuel Kaski

TL;DR
This paper provides a theoretical foundation for discriminative posterior inference within Bayesian models, showing its consistency and advantages over generative approaches, especially with small or incorrect models.
Contribution
It offers an axiomatic proof of discriminative posterior consistency and extends its practical use to handle missing data and incorrect models.
Findings
Discriminative posterior is consistent for conditional inference.
It performs better than generative methods when models are incorrect.
It aligns with Bayesian regression practice and can handle missing data.
Abstract
We study Bayesian discriminative inference given a model family that is assumed to contain all our prior information but still known to be incorrect. This falls in between "standard" Bayesian generative modeling and Bayesian regression, where the margin is known to be uninformative about . We give an axiomatic proof that discriminative posterior is consistent for conditional inference; using the discriminative posterior is standard practice in classical Bayesian regression, but we show that it is theoretically justified for model families of joint densities as well. A practical benefit compared to Bayesian regression is that the standard methods of handling missing values in generative modeling can be extended into discriminative inference, which is useful if the amount of data is small. Compared to standard generative modeling,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsBayesian Methods and Mixture Models · Gaussian Processes and Bayesian Inference · Bayesian Modeling and Causal Inference
