Probabilistic Model-Agnostic Meta-Learning

TL;DR

This paper introduces a probabilistic extension to model-agnostic meta-learning that enables sampling diverse models for ambiguous few-shot tasks, improving uncertainty modeling and active learning applications.

Contribution

It extends MAML by incorporating a parameter distribution trained with a variational lower bound, allowing sampling of models and better handling of task ambiguity.

Findings

Samples plausible classifiers and regressors in ambiguous few-shot tasks.

Enhances active learning by reasoning about task ambiguity.

Demonstrates improved uncertainty modeling in meta-learning.

Abstract

Meta-learning for few-shot learning entails acquiring a prior over previous tasks and experiences, such that new tasks be learned from small amounts of data. However, a critical challenge in few-shot learning is task ambiguity: even when a powerful prior can be meta-learned from a large number of prior tasks, a small dataset for a new task can simply be too ambiguous to acquire a single model (e.g., a classifier) for that task that is accurate. In this paper, we propose a probabilistic meta-learning algorithm that can sample models for a new task from a model distribution. Our approach extends model-agnostic meta-learning, which adapts to new tasks via gradient descent, to incorporate a parameter distribution that is trained via a variational lower bound. At meta-test time, our algorithm adapts via a simple procedure that injects noise into gradient descent, and at meta-training time, the model is trained such that this stochastic adaptation procedure produces samples from the approximate model posterior. Our experimental results show that our method can sample plausible classifiers and regressors in ambiguous few-shot learning problems. We also show how reasoning about ambiguity can also be used for downstream active learning problems.