Fast Context Adaptation via Meta-Learning

TL;DR

CAVIA is a meta-learning method that improves upon MAML by using adaptable context parameters for better task-specific adaptation, reduced overfitting, and enhanced interpretability across various learning tasks.

Contribution

CAVIA introduces a simple extension to MAML that partitions parameters into context and shared parts, improving adaptation efficiency and interpretability.

Findings

CAVIA outperforms MAML in regression, classification, and reinforcement learning.

CAVIA is less prone to meta-overfitting and easier to parallelize.

Current benchmarks may underestimate the adaptation needed in some tasks.

Abstract

We propose CAVIA for meta-learning, a simple extension to MAML that is less prone to meta-overfitting, easier to parallelise, and more interpretable. CAVIA partitions the model parameters into two parts: context parameters that serve as additional input to the model and are adapted on individual tasks, and shared parameters that are meta-trained and shared across tasks. At test time, only the context parameters are updated, leading to a low-dimensional task representation. We show empirically that CAVIA outperforms MAML for regression, classification, and reinforcement learning. Our experiments also highlight weaknesses in current benchmarks, in that the amount of adaptation needed in some cases is small.