Continuous Adversarial Text Representation Learning for Affective Recognition

TL;DR

This paper introduces a novel framework that enhances emotion-aware embeddings in transformer models by using continuous valence-arousal labels and dynamic token perturbation, significantly improving affective recognition performance.

Contribution

The paper presents a new approach combining continuous affective labels with contrastive learning and saliency-guided token perturbation for better emotion representation in language models.

Findings

Achieved up to 15.5% improvement in emotion classification accuracy.

Effectively captures subtle emotional nuances with continuous labels.

Enhances model sensitivity to sentiment-relevant tokens.

Abstract

While pre-trained language models excel at semantic understanding, they often struggle to capture nuanced affective information critical for affective recognition tasks. To address these limitations, we propose a novel framework for enhancing emotion-aware embeddings in transformer-based models. Our approach introduces a continuous valence-arousal labeling system to guide contrastive learning, which captures subtle and multi-dimensional emotional nuances more effectively. Furthermore, we employ a dynamic token perturbation mechanism, using gradient-based saliency to focus on sentiment-relevant tokens, improving model sensitivity to emotional cues. The experimental results demonstrate that the proposed framework outperforms existing methods, achieving up to 15.5% improvement in the emotion classification benchmark, highlighting the importance of employing continuous labels. This improvement demonstrates that the proposed framework is effective in affective representation learning and enables precise and contextually relevant emotional understanding.