GUDN: A novel guide network with label reinforcement strategy for extreme multi-label text classification

TL;DR

This paper introduces GUDN, a guide network with label reinforcement, to enhance fine-tuning of pre-trained models for extreme multi-label text classification by bridging the semantic gap between texts and labels.

Contribution

The paper proposes a novel guide network and label reinforcement strategy to improve fine-tuning in XMTC, addressing the semantic gap issue and achieving superior accuracy.

Findings

GUDN outperforms state-of-the-art methods on Eurlex-4k.

The label reinforcement strategy effectively narrows the semantic gap.

Input length influences Transformer model accuracy.

Abstract

In natural language processing, extreme multi-label text classification is an emerging but essential task. The problem of extreme multi-label text classification (XMTC) is to recall some of the most relevant labels for a text from an extremely large label set. Large-scale pre-trained models have brought a new trend to this problem. Though the large-scale pre-trained models have made significant achievements on this problem, the valuable fine-tuned methods have yet to be studied. Though label semantics have been introduced in XMTC, the vast semantic gap between texts and labels has yet to gain enough attention. This paper builds a new guide network (GUDN) to help fine-tune the pre-trained model to instruct classification later. Furthermore, GUDN uses raw label semantics combined with a helpful label reinforcement strategy to effectively explore the latent space between texts and labels, narrowing the semantic gap, which can further improve predicted accuracy. Experimental results demonstrate that GUDN outperforms state-of-the-art methods on Eurlex-4k and has competitive results on other popular datasets. In an additional experiment, we investigated the input lengths' influence on the Transformer-based model's accuracy. Our source code is released at https://t.hk.uy/aFSH.