Diversity Over Quantity: A Lesson From Few Shot Relation Classification

TL;DR

This paper emphasizes that in few-shot relation classification, increasing the diversity of relation types in training data improves generalization to unseen relations more effectively than merely increasing dataset size.

Contribution

The authors introduce REBEL-FS, a new benchmark with more diverse relation types, and demonstrate that relation diversity enhances FSRC performance.

Findings

Diverse relation training data improves generalization to unseen relations.

REBEL-FS contains an order of magnitude more relation types than existing datasets.

Diversity-focused data curation reduces the need for large datasets in FSRC.

Abstract

In few-shot relation classification (FSRC), models must generalize to novel relations with only a few labeled examples. While much of the recent progress in NLP has focused on scaling data size, we argue that diversity in relation types is more crucial for FSRC performance. In this work, we demonstrate that training on a diverse set of relations significantly enhances a model's ability to generalize to unseen relations, even when the overall dataset size remains fixed. We introduce REBEL-FS, a new FSRC benchmark that incorporates an order of magnitude more relation types than existing datasets. Through systematic experiments, we show that increasing the diversity of relation types in the training data leads to consistent gains in performance across various few-shot learning scenarios, including high-negative settings. Our findings challenge the common assumption that more data alone leads to better performance and suggest that targeted data curation focused on diversity can substantially reduce the need for large-scale datasets in FSRC.