Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning

TL;DR

This paper introduces a method to adapt pretrained MAML checkpoints for new few-shot classification tasks, addressing distribution mismatch and improving robustness without requiring access to original meta-training data.

Contribution

The authors propose a novel meta-testing procedure combining MAML gradient steps with adversarial training and uncertainty-based stepsize adaptation for better out-of-domain few-shot learning.

Findings

Outperforms vanilla MAML on same-domain benchmarks

Shows improved robustness to stepsize choices

Effective in cross-domain scenarios

Abstract

Model-agnostic meta-learning (MAML) is a popular method for few-shot learning but assumes that we have access to the meta-training set. In practice, training on the meta-training set may not always be an option due to data privacy concerns, intellectual property issues, or merely lack of computing resources. In this paper, we consider the novel problem of repurposing pretrained MAML checkpoints to solve new few-shot classification tasks. Because of the potential distribution mismatch, the original MAML steps may no longer be optimal. Therefore we propose an alternative meta-testing procedure and combine MAML gradient steps with adversarial training and uncertainty-based stepsize adaptation. Our method outperforms "vanilla" MAML on same-domain and cross-domains benchmarks using both SGD and Adam optimizers and shows improved robustness to the choice of base stepsize.