Class-Aware Contrastive Optimization for Imbalanced Text Classification

TL;DR

This paper introduces a class-aware contrastive optimization method combined with autoencoders to improve imbalanced text classification, outperforming existing approaches by enhancing class separation in embeddings.

Contribution

The paper proposes a novel combination of contrastive loss and autoencoder reconstruction to better handle class imbalance in text classification tasks.

Findings

Significant performance improvement over state-of-the-art methods

Effective class separation in embedding space

Robust across various text datasets

Abstract

The unique characteristics of text data make classification tasks a complex problem. Advances in unsupervised and semi-supervised learning and autoencoder architectures addressed several challenges. However, they still struggle with imbalanced text classification tasks, a common scenario in real-world applications, demonstrating a tendency to produce embeddings with unfavorable properties, such as class overlap. In this paper, we show that leveraging class-aware contrastive optimization combined with denoising autoencoders can successfully tackle imbalanced text classification tasks, achieving better performance than the current state-of-the-art. Concretely, our proposal combines reconstruction loss with contrastive class separation in the embedding space, allowing a better balance between the truthfulness of the generated embeddings and the model's ability to separate different classes. Compared with an extensive set of traditional and state-of-the-art competing methods, our proposal demonstrates a notable increase in performance across a wide variety of text datasets.