Hierarchical Verbalizer for Few-Shot Hierarchical Text Classification

TL;DR

This paper introduces HierVerb, a hierarchical verbalizer framework that enhances few-shot hierarchical text classification by integrating label hierarchy knowledge into prompt-based learning with pre-trained language models.

Contribution

The paper proposes HierVerb, a novel multi-verbalizer approach that incorporates hierarchical structure and contrastive learning for improved few-shot HTC performance.

Findings

HierVerb outperforms graph encoder-based hierarchy injection methods.

Prompt-based HTC with HierVerb significantly improves accuracy in few-shot settings.

The approach effectively bridges the gap between PLMs and hierarchical classification tasks.

Abstract

Due to the complex label hierarchy and intensive labeling cost in practice, the hierarchical text classification (HTC) suffers a poor performance especially when low-resource or few-shot settings are considered. Recently, there is a growing trend of applying prompts on pre-trained language models (PLMs), which has exhibited effectiveness in the few-shot flat text classification tasks. However, limited work has studied the paradigm of prompt-based learning in the HTC problem when the training data is extremely scarce. In this work, we define a path-based few-shot setting and establish a strict path-based evaluation metric to further explore few-shot HTC tasks. To address the issue, we propose the hierarchical verbalizer ("HierVerb"), a multi-verbalizer framework treating HTC as a single- or multi-label classification problem at multiple layers and learning vectors as verbalizers constrained by hierarchical structure and hierarchical contrastive learning. In this manner, HierVerb fuses label hierarchy knowledge into verbalizers and remarkably outperforms those who inject hierarchy through graph encoders, maximizing the benefits of PLMs. Extensive experiments on three popular HTC datasets under the few-shot settings demonstrate that prompt with HierVerb significantly boosts the HTC performance, meanwhile indicating an elegant way to bridge the gap between the large pre-trained model and downstream hierarchical classification tasks. Our code and few-shot dataset are publicly available at https://github.com/1KE-JI/HierVerb.