# SPIRIT: Structural Entropy Guided Prefix Tuning for Hierarchical Text Classification

## TL;DR

This paper introduces SPIRIT, a new method for hierarchical text classification that uses structural entropy to improve model performance.

## Contribution

SPIRIT introduces a novel prefix tuning approach guided by structural entropy for hierarchical text classification.

## Key findings

- SPIRIT outperforms existing methods on four popular hierarchical text classification datasets.
- The method uses structural entropy to capture the essential hierarchy of labels.
- A depth-wise reparameterization strategy enhances optimization and model performance.

## Abstract

Hierarchical text classification (HTC) is a challenging task that requires classifiers to solve a series of multi-label subtasks considering hierarchical dependencies among labels. Recent studies have introduced prompt tuning to create closer connections between the language model (LM) and the complex label hierarchy. However, we find that the model’s attention to the prompt gradually decreases as the prompt moves from the input to the output layer, revealing the limitations of previous prompt tuning methods for HTC. Given the success of prefix tuning-based studies in natural language understanding tasks, we introduce Structural entroPy guIded pRefIx Tuning (SPIRIT). Specifically, we extract the essential structure of the label hierarchy via structural entropy minimization and decode the abstractive structural information as the prefix to prompt all intermediate layers in the LM. Additionally, a depth-wise reparameterization strategy is developed to enhance optimization and propagate the prefix throughout the LM layers. Extensive evaluation on four popular datasets demonstrates that SPIRIT achieves a state-of-the-art performance.