An Effective Deployment of Contrastive Learning in Multi-label Text Classification

TL;DR

This paper introduces five novel contrastive loss functions tailored for multi-label text classification, demonstrating their effectiveness through experiments and providing insights into their roles and deployment strategies.

Contribution

It proposes new contrastive loss functions specifically designed for multi-label text classification and offers baseline models and analysis for their application.

Findings

Proposed five novel contrastive losses for multi-label classification.

Contrastive losses improve performance on multi-label text classification tasks.

Provided interpretability analysis of contrastive learning components.

Abstract

The effectiveness of contrastive learning technology in natural language processing tasks is yet to be explored and analyzed. How to construct positive and negative samples correctly and reasonably is the core challenge of contrastive learning. It is even harder to discover contrastive objects in multi-label text classification tasks. There are very few contrastive losses proposed previously. In this paper, we investigate the problem from a different angle by proposing five novel contrastive losses for multi-label text classification tasks. These are Strict Contrastive Loss (SCL), Intra-label Contrastive Loss (ICL), Jaccard Similarity Contrastive Loss (JSCL), Jaccard Similarity Probability Contrastive Loss (JSPCL), and Stepwise Label Contrastive Loss (SLCL). We explore the effectiveness of contrastive learning for multi-label text classification tasks by the employment of these novel losses and provide a set of baseline models for deploying contrastive learning techniques on specific tasks. We further perform an interpretable analysis of our approach to show how different components of contrastive learning losses play their roles. The experimental results show that our proposed contrastive losses can bring improvement to multi-label text classification tasks. Our work also explores how contrastive learning should be adapted for multi-label text classification tasks.