Harnessing the Power of Local Representations for Few-Shot Classification

TL;DR

This paper introduces a novel approach for few-shot classification that leverages local representations through a specialized pretraining paradigm and an adaptable metric, achieving state-of-the-art results.

Contribution

It proposes a new pretraining method with soft labels and a flexible metric based on optimal transport, enhancing generalization to novel classes.

Findings

Achieves new state-of-the-art performance on three benchmarks.

Outperforms existing methods in fine-grained scenarios.

Effectively handles homogeneous local feature sets with the proposed metric.

Abstract

Generalizing to novel classes unseen during training is a key challenge of few-shot classification. Recent metric-based methods try to address this by local representations. However, they are unable to take full advantage of them due to (i) improper supervision for pretraining the feature extractor, and (ii) lack of adaptability in the metric for handling various possible compositions of local feature sets. In this work, we harness the power of local representations in improving novel-class generalization. For the feature extractor, we design a novel pretraining paradigm that learns randomly cropped patches by soft labels. It utilizes the class-level diversity of patches while diminishing the impact of their semantic misalignments to hard labels. To align network output with soft labels, we also propose a UniCon KL-Divergence that emphasizes the equal contribution of each base class in describing "non-base" patches. For the metric, we formulate measuring local feature sets as an entropy-regularized optimal transport problem to introduce the ability to handle sets consisting of homogeneous elements. Furthermore, we design a Modulate Module to endow the metric with the necessary adaptability. Our method achieves new state-of-the-art performance on three popular benchmarks. Moreover, it exceeds state-of-the-art transductive and cross-modal methods in the fine-grained scenario.