Revisiting Few-Shot Learning from a Causal Perspective

TL;DR

This paper offers a causal perspective on metric-based few-shot learning methods, interpreting them as front-door adjustments, and introduces two new causal methods that improve classification performance on benchmarks.

Contribution

It provides a novel causal interpretation of existing metric-based few-shot learning methods and proposes two new causal approaches that enhance classification accuracy.

Findings

Causal interpretation explains success of existing methods

Proposed causal methods outperform baselines on benchmarks

Diversity of representations improves few-shot learning

Abstract

Few-shot learning with $N$-way $K$-shot scheme is an open challenge in machine learning. Many metric-based approaches have been proposed to tackle this problem, e.g., the Matching Networks and CLIP-Adapter. Despite that these approaches have shown significant progress, the mechanism of why these methods succeed has not been well explored. In this paper, we try to interpret these metric-based few-shot learning methods via causal mechanism. We show that the existing approaches can be viewed as specific forms of front-door adjustment, which can alleviate the effect of spurious correlations and thus learn the causality. This causal interpretation could provide us a new perspective to better understand these existing metric-based methods. Further, based on this causal interpretation, we simply introduce two causal methods for metric-based few-shot learning, which considers not only the relationship between examples but also the diversity of representations. Experimental results demonstrate the superiority of our proposed methods in few-shot classification on various benchmark datasets. Code is available in https://github.com/lingl1024/causalFewShot.