CLLD: Contrastive Learning with Label Distance for Text Classification

TL;DR

This paper introduces CLLD, a contrastive learning approach that incorporates label distance to better distinguish similar texts and improve text classification performance.

Contribution

The paper proposes a novel contrastive learning method with label distance to enhance class distinction in text classification tasks.

Findings

Improves pre-trained model performance on classification benchmarks.

Relieves adversarial interclass relationships.

Effective in distinguishing subtle semantic differences.

Abstract

Existed pre-trained models have achieved state-of-the-art performance on various text classification tasks. These models have proven to be useful in learning universal language representations. However, the semantic discrepancy between similar texts cannot be effectively distinguished by advanced pre-trained models, which have a great influence on the performance of hard-to-distinguish classes. To address this problem, we propose a novel Contrastive Learning with Label Distance (CLLD) in this work. Inspired by recent advances in contrastive learning, we specifically design a classification method with label distance for learning contrastive classes. CLLD ensures the flexibility within the subtle differences that lead to different label assignments, and generates the distinct representations for each class having similarity simultaneously. Extensive experiments on public benchmarks and internal datasets demonstrate that our method improves the performance of pre-trained models on classification tasks. Importantly, our experiments suggest that the learned label distance relieve the adversarial nature of interclasses.