Less is More: A Closer Look at Semantic-based Few-Shot Learning

TL;DR

This paper introduces a straightforward framework for semantic-based few-shot learning that leverages pre-trained language models and textual information, significantly improving performance especially in 1-shot scenarios.

Contribution

It explicitly exploits the zero-shot capability of language models with learnable prompts and combines visual and textual features directly, avoiding complex fusion modules used previously.

Findings

Achieves state-of-the-art results in 1-shot learning, surpassing previous methods by 3.0% accuracy.

Demonstrates effectiveness across four widely used few-shot datasets.

Utilizes self-ensemble and distillation techniques to enhance model performance.

Abstract

Few-shot Learning aims to learn and distinguish new categories with a very limited number of available images, presenting a significant challenge in the realm of deep learning. Recent researchers have sought to leverage the additional textual or linguistic information of these rare categories with a pre-trained language model to facilitate learning, thus partially alleviating the problem of insufficient supervision signals. However, the full potential of the textual information and pre-trained language model have been underestimated in the few-shot learning till now, resulting in limited performance enhancements. To address this, we propose a simple but effective framework for few-shot learning tasks, specifically designed to exploit the textual information and language model. In more detail, we explicitly exploit the zero-shot capability of the pre-trained language model with the learnable prompt. And we just add the visual feature with the textual feature for inference directly without the intricate designed fusion modules in previous works. Additionally, we apply the self-ensemble and distillation to further enhance these components. Our extensive experiments conducted across four widely used few-shot datasets demonstrate that our simple framework achieves impressive results. Particularly noteworthy is its outstanding performance in the 1-shot learning task, surpassing state-of-the-art methods by an average of 3.0\% in classification accuracy. \footnote{We will make the source codes of the proposed framework publicly available upon acceptance. }.