Contrastive Knowledge-Augmented Meta-Learning for Few-Shot Classification

TL;DR

CAML introduces a knowledge graph and contrastive distillation to enhance few-shot learning, especially under challenging distribution shifts, outperforming existing methods across multiple benchmarks.

Contribution

The paper proposes CAML, a novel meta-learning framework that uses a knowledge graph and contrastive learning to improve adaptation in diverse and challenging few-shot tasks.

Findings

CAML outperforms existing methods on standard benchmarks.

It improves generalization in multi-domain and dataset generalization scenarios.

CAML effectively encodes historical experience for better task adaptation.

Abstract

Model agnostic meta-learning algorithms aim to infer priors from several observed tasks that can then be used to adapt to a new task with few examples. Given the inherent diversity of tasks arising in existing benchmarks, recent methods use separate, learnable structure, such as hierarchies or graphs, for enabling task-specific adaptation of the prior. While these approaches have produced significantly better meta learners, our goal is to improve their performance when the heterogeneous task distribution contains challenging distribution shifts and semantic disparities. To this end, we introduce CAML (Contrastive Knowledge-Augmented Meta Learning), a novel approach for knowledge-enhanced few-shot learning that evolves a knowledge graph to effectively encode historical experience, and employs a contrastive distillation strategy to leverage the encoded knowledge for task-aware modulation of the base learner. Using standard benchmarks, we evaluate the performance of CAML in different few-shot learning scenarios. In addition to the standard few-shot task adaptation, we also consider the more challenging multi-domain task adaptation and few-shot dataset generalization settings in our empirical studies. Our results shows that CAML consistently outperforms best known approaches and achieves improved generalization.