Bayesian Deconditional Kernel Mean Embeddings

TL;DR

This paper introduces a Bayesian framework for deconditional kernel mean embeddings, enabling inverse inference, uncertainty quantification, and hyperparameter learning, with applications in likelihood-free inference and large-scale data analysis.

Contribution

It formalizes deconditional kernel mean embeddings as a nonparametric Bayes' rule and links them to task transformed Gaussian processes, providing Bayesian interpretation and practical tools.

Findings

Deconditional kernel mean embeddings can be viewed as posterior means of task transformed Gaussian processes.

The approach offers uncertainty estimates and hyperparameter learning via marginal likelihood.

Applications include likelihood-free inference and scalable learning for big data.

Abstract

Conditional kernel mean embeddings form an attractive nonparametric framework for representing conditional means of functions, describing the observation processes for many complex models. However, the recovery of the original underlying function of interest whose conditional mean was observed is a challenging inference task. We formalize deconditional kernel mean embeddings as a solution to this inverse problem, and show that it can be naturally viewed as a nonparametric Bayes' rule. Critically, we introduce the notion of task transformed Gaussian processes and establish deconditional kernel means as their posterior predictive mean. This connection provides Bayesian interpretations and uncertainty estimates for deconditional kernel mean embeddings, explains their regularization hyperparameters, and reveals a marginal likelihood for kernel hyperparameter learning. These revelations further enable practical applications such as likelihood-free inference and learning sparse representations for big data.