Self-Supervision Can Be a Good Few-Shot Learner
Yuning Lu, Liangjian Wen, Jianzhuang Liu, Yajing Liu, Xinmei Tian

TL;DR
This paper introduces a self-supervised learning approach for few-shot learning that leverages unlabeled data to improve generalization to unseen classes, outperforming supervised pre-training in certain conditions.
Contribution
It proposes an unsupervised FSL method based on maximizing mutual information via self-supervision, which reduces bias towards seen classes and enhances performance on unseen classes.
Findings
Self-supervised pre-training can outperform supervised pre-training under certain conditions.
The method achieves comparable results to state-of-the-art FSL methods without using labeled base class data.
Extensive experiments validate the effectiveness of the proposed approach.
Abstract
Existing few-shot learning (FSL) methods rely on training with a large labeled dataset, which prevents them from leveraging abundant unlabeled data. From an information-theoretic perspective, we propose an effective unsupervised FSL method, learning representations with self-supervision. Following the InfoMax principle, our method learns comprehensive representations by capturing the intrinsic structure of the data. Specifically, we maximize the mutual information (MI) of instances and their representations with a low-bias MI estimator to perform self-supervised pre-training. Rather than supervised pre-training focusing on the discriminable features of the seen classes, our self-supervised model has less bias toward the seen classes, resulting in better generalization for unseen classes. We explain that supervised pre-training and self-supervised pre-training are actually maximizing…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsDomain Adaptation and Few-Shot Learning · Machine Learning and Data Classification · Text and Document Classification Technologies
