Variational Prediction

TL;DR

This paper introduces variational prediction, a method for approximating the posterior predictive distribution directly, reducing computational costs during testing while maintaining good predictive performance.

Contribution

It proposes a novel variational approach to directly learn the posterior predictive distribution, bypassing costly marginalization at test time.

Findings

Effective on a toy example

Reduces test-time computational costs

Maintains good predictive accuracy

Abstract

Bayesian inference offers benefits over maximum likelihood, but it also comes with computational costs. Computing the posterior is typically intractable, as is marginalizing that posterior to form the posterior predictive distribution. In this paper, we present variational prediction, a technique for directly learning a variational approximation to the posterior predictive distribution using a variational bound. This approach can provide good predictive distributions without test time marginalization costs. We demonstrate Variational Prediction on an illustrative toy example.