Rethinking the Good Enough Embedding for Easy Few-Shot Learning

TL;DR

This paper argues that pre-trained off-the-shelf embeddings are sufficient for few-shot learning, eliminating the need for complex fine-tuning, and achieves state-of-the-art results with a simple non-parametric approach.

Contribution

It introduces a non-parametric, fine-tuning-free pipeline using frozen features and simple manifold refinement, outperforming existing meta-learning methods in few-shot tasks.

Findings

Using frozen DINOv2-L features with a k-NN classifier yields strong few-shot performance.

Manifold refinement with PCA and ICA improves the quality of embeddings.

The approach surpasses sophisticated meta-learning algorithms on four benchmarks.

Abstract

The field of deep visual recognition is undergoing a paradigm shift toward universal representations. The Platonic Representation Hypothesis suggests that diverse architectures trained on massive datasets are converging toward a shared, "ideal" latent space. This again raises a critical question: is a "Good Embedding All You Need?" In this paper, we leverage this convergence to demonstrate that off-the-shelf embeddings are inherently "good enough" for complex tasks, rendering intensive task-specific fine-tuning unnecessary. We explore this hypothesis within the few-shot learning framework, proposing a straightforward, non-parametric pipeline that entirely bypasses backpropagation. By utilizing a k-Nearest Neighbor classifier on frozen DINOv2-L features, we conduct a layer-wise characterization to identify an optimal feature extraction. We further demonstrate that manifold refinement via PCA and ICA provides a beneficial regularizing effect. Our results across four major benchmarks demonstrate that our approach consistently surpasses sophisticated meta-learning algorithms, achieving state-of-the-art performance.