Emotions are Subtle: Learning Sentiment Based Text Representations Using Contrastive Learning

TL;DR

This paper applies contrastive learning to generate sentiment-aware text embeddings, demonstrating improved sentiment analysis performance and cross-domain robustness over traditional BERT embeddings.

Contribution

It introduces contrastive learning for sentiment-based text representations and shows their effectiveness in improving sentiment analysis benchmarks.

Findings

Contrastive learning embeddings outperform BERT in sentiment analysis.

Fine-tuning on these embeddings yields higher benchmark scores.

Upsampling techniques further improve class balance and performance.

Abstract

Contrastive learning techniques have been widely used in the field of computer vision as a means of augmenting datasets. In this paper, we extend the use of these contrastive learning embeddings to sentiment analysis tasks and demonstrate that fine-tuning on these embeddings provides an improvement over fine-tuning on BERT-based embeddings to achieve higher benchmarks on the task of sentiment analysis when evaluated on the DynaSent dataset. We also explore how our fine-tuned models perform on cross-domain benchmark datasets. Additionally, we explore upsampling techniques to achieve a more balanced class distribution to make further improvements on our benchmark tasks.