Bayesian Meta-Learning for the Few-Shot Setting via Deep Kernels

TL;DR

This paper introduces Deep Kernel Transfer (DKT), a Bayesian meta-learning method using deep kernels that improves few-shot learning by transferring kernels to new tasks, offering uncertainty quantification and simplicity.

Contribution

The paper proposes a Bayesian meta-learning approach with deep kernels, simplifying the process and outperforming state-of-the-art methods in few-shot classification and cross-domain tasks.

Findings

DKT outperforms several state-of-the-art algorithms in few-shot classification.

DKT is the state of the art for cross-domain adaptation and regression.

The approach provides uncertainty quantification without task-specific parameter estimation.

Abstract

Recently, different machine learning methods have been introduced to tackle the challenging few-shot learning scenario that is, learning from a small labeled dataset related to a specific task. Common approaches have taken the form of meta-learning: learning to learn on the new problem given the old. Following the recognition that meta-learning is implementing learning in a multi-level model, we present a Bayesian treatment for the meta-learning inner loop through the use of deep kernels. As a result we can learn a kernel that transfers to new tasks; we call this Deep Kernel Transfer (DKT). This approach has many advantages: is straightforward to implement as a single optimizer, provides uncertainty quantification, and does not require estimation of task-specific parameters. We empirically demonstrate that DKT outperforms several state-of-the-art algorithms in few-shot classification, and is the state of the art for cross-domain adaptation and regression. We conclude that complex meta-learning routines can be replaced by a simpler Bayesian model without loss of accuracy.