Modeling Text-Label Alignment for Hierarchical Text Classification

TL;DR

This paper introduces a novel contrastive learning-based Text-Label Alignment (TLA) loss and a Hierarchical Text-Label Alignment (HTLA) model that improve hierarchical text classification by modeling dynamic text-label relationships.

Contribution

It proposes a new TLA loss for better text-label alignment and a HTLA model that leverages BERT and graph encoders for improved hierarchical classification.

Findings

HTLA outperforms existing baselines on benchmark datasets.

The TLA loss effectively aligns text and label representations.

The model captures hierarchy-aware features for classification.

Abstract

Hierarchical Text Classification (HTC) aims to categorize text data based on a structured label hierarchy, resulting in predicted labels forming a sub-hierarchy tree. The semantics of the text should align with the semantics of the labels in this sub-hierarchy. With the sub-hierarchy changing for each sample, the dynamic nature of text-label alignment poses challenges for existing methods, which typically process text and labels independently. To overcome this limitation, we propose a Text-Label Alignment (TLA) loss specifically designed to model the alignment between text and labels. We obtain a set of negative labels for a given text and its positive label set. By leveraging contrastive learning, the TLA loss pulls the text closer to its positive label and pushes it away from its negative label in the embedding space. This process aligns text representations with related labels while distancing them from unrelated ones. Building upon this framework, we introduce the Hierarchical Text-Label Alignment (HTLA) model, which leverages BERT as the text encoder and GPTrans as the graph encoder and integrates text-label embeddings to generate hierarchy-aware representations. Experimental results on benchmark datasets and comparison with existing baselines demonstrate the effectiveness of HTLA for HTC.