Few-shot Classification with Hypersphere Modeling of Prototypes

TL;DR

This paper introduces HyperProto, a hypersphere-based prototype method for few-shot learning that models classes as hyperspheres, offering greater expressivity and simplicity over traditional embedding or statistical approaches.

Contribution

The paper proposes hypersphere prototypes (HyperProto) for few-shot learning, extending class representations from points to areas for improved expressivity and easier metric-based classification.

Findings

HyperProto outperforms 20+ baselines in NLP and CV tasks.

Hypersphere modeling improves class representation robustness.

The method simplifies metric calculations in few-shot learning.

Abstract

Metric-based meta-learning is one of the de facto standards in few-shot learning. It composes of representation learning and metrics calculation designs. Previous works construct class representations in different ways, varying from mean output embedding to covariance and distributions. However, using embeddings in space lacks expressivity and cannot capture class information robustly, while statistical complex modeling poses difficulty to metric designs. In this work, we use tensor fields (``areas'') to model classes from the geometrical perspective for few-shot learning. We present a simple and effective method, dubbed hypersphere prototypes (HyperProto), where class information is represented by hyperspheres with dynamic sizes with two sets of learnable parameters: the hypersphere's center and the radius. Extending from points to areas, hyperspheres are much more expressive than embeddings. Moreover, it is more convenient to perform metric-based classification with hypersphere prototypes than statistical modeling, as we only need to calculate the distance from a data point to the surface of the hypersphere. Following this idea, we also develop two variants of prototypes under other measurements. Extensive experiments and analysis on few-shot learning tasks across NLP and CV and comparison with 20+ competitive baselines demonstrate the effectiveness of our approach.