FewShotTextGCN: K-hop neighborhood regularization for few-shot learning on graphs

TL;DR

FewShotTextGCN introduces a graph-based approach with neighborhood regularization and graph simplification to significantly improve low-resource text classification across multiple languages, outperforming baseline models.

Contribution

The paper proposes FewShotTextGCN, a novel method with K-hop regularization and graph simplification, enabling effective low-resource text classification without language model pretraining.

Findings

Outperforms baseline by 17% accuracy with only 20 samples.

Effective across eight typologically diverse languages.

Achieves comparable results to large pretrained language models.

Abstract

We present FewShotTextGCN, a novel method designed to effectively utilize the properties of word-document graphs for improved learning in low-resource settings. We introduce K-hop Neighbourhood Regularization, a regularizer for heterogeneous graphs, and show that it stabilizes and improves learning when only a few training samples are available. We furthermore propose a simplification in the graph-construction method, which results in a graph that is $\sim$7 times less dense and yields better performance in little-resource settings while performing on par with the state of the art in high-resource settings. Finally, we introduce a new variant of Adaptive Pseudo-Labeling tailored for word-document graphs. When using as little as 20 samples for training, we outperform a strong TextGCN baseline with 17% in absolute accuracy on average over eight languages. We demonstrate that our method can be applied to document classification without any language model pretraining on a wide range of typologically diverse languages while performing on par with large pretrained language models.