Bayesian Meta-Learning Through Variational Gaussian Processes

TL;DR

This paper introduces VMGP, a Bayesian meta-learning method based on Gaussian processes that can model complex, non-Gaussian uncertainties, outperforming existing methods on challenging benchmarks.

Contribution

We develop VMGP, extending Gaussian-process meta-learning to produce high-quality, non-Gaussian predictive uncertainties, addressing limitations of traditional Gaussian process models.

Findings

VMGP outperforms existing Bayesian meta-learning baselines on complex benchmarks.

The method effectively models non-Gaussian, discontinuous uncertainty structures.

Experimental results demonstrate significant improvements in predictive accuracy and uncertainty quantification.

Abstract

Recent advances in the field of meta-learning have tackled domains consisting of large numbers of small ("few-shot") supervised learning tasks. Meta-learning algorithms must be able to rapidly adapt to any individual few-shot task, fitting to a small support set within a task and using it to predict the labels of the task's query set. This problem setting can be extended to the Bayesian context, wherein rather than predicting a single label for each query data point, a model predicts a distribution of labels capturing its uncertainty. Successful methods in this domain include Bayesian ensembling of MAML-based models, Bayesian neural networks, and Gaussian processes with learned deep kernel and mean functions. While Gaussian processes have a robust Bayesian interpretation in the meta-learning context, they do not naturally model non-Gaussian predictive posteriors for expressing uncertainty. In this paper, we design a theoretically principled method, VMGP, extending Gaussian-process-based meta-learning to allow for high-quality, arbitrary non-Gaussian uncertainty predictions. On benchmark environments with complex non-smooth or discontinuous structure, we find our VMGP method performs significantly better than existing Bayesian meta-learning baselines.