Hierarchical Text Classification As Sub-Hierarchy Sequence Generation

TL;DR

This paper introduces HiDEC, a novel hierarchical text classification model that reformulates the task as sub-hierarchy sequence generation, enabling efficient large-scale classification with fewer parameters and achieving state-of-the-art results.

Contribution

The paper proposes a sub-hierarchy sequence generation approach with the Hierarchy DECoder (HiDEC), improving scalability and performance in hierarchical text classification.

Findings

Achieved state-of-the-art performance on RCV1-v2, NYT, and EURLEX57K datasets.

Significantly reduced model parameters compared to existing models.

Effectively incorporates hierarchy information using recursive decoding and attention mechanisms.

Abstract

Hierarchical text classification (HTC) is essential for various real applications. However, HTC models are challenging to develop because they often require processing a large volume of documents and labels with hierarchical taxonomy. Recent HTC models based on deep learning have attempted to incorporate hierarchy information into a model structure. Consequently, these models are challenging to implement when the model parameters increase for a large-scale hierarchy because the model structure depends on the hierarchy size. To solve this problem, we formulate HTC as a sub-hierarchy sequence generation to incorporate hierarchy information into a target label sequence instead of the model structure. Subsequently, we propose the Hierarchy DECoder (HiDEC), which decodes a text sequence into a sub-hierarchy sequence using recursive hierarchy decoding, classifying all parents at the same level into children at once. In addition, HiDEC is trained to use hierarchical path information from a root to each leaf in a sub-hierarchy composed of the labels of a target document via an attention mechanism and hierarchy-aware masking. HiDEC achieved state-of-the-art performance with significantly fewer model parameters than existing models on benchmark datasets, such as RCV1-v2, NYT, and EURLEX57K.