Dynamic Adaptive Attention and Supervised Contrastive Learning: A Novel Hybrid Framework for Text Sentiment Classification

TL;DR

This paper introduces a hybrid model combining dynamic adaptive attention and supervised contrastive learning within a BERT framework to improve sentiment classification accuracy on lengthy reviews.

Contribution

It presents a novel hybrid framework that enhances attention focus and embedding space separation, outperforming existing models on the IMDB dataset.

Findings

Achieved 94.67% accuracy on IMDB dataset

Outperformed strong baselines by 1.5-2.5 percentage points

Framework is lightweight, efficient, and extensible

Abstract

The exponential growth of user-generated movie reviews on digital platforms has made accurate text sentiment classification a cornerstone task in natural language processing. Traditional models, including standard BERT and recurrent architectures, frequently struggle to capture long-distance semantic dependencies and resolve ambiguous emotional expressions in lengthy review texts. This paper proposes a novel hybrid framework that seamlessly integrates dynamic adaptive multi-head attention with supervised contrastive learning into a BERT-based Transformer encoder. The dynamic adaptive attention module employs a global context pooling vector to dynamically regulate the contribution of each attention head, thereby focusing on critical sentiment-bearing tokens while suppressing noise. Simultaneously, the supervised contrastive learning branch enforces tighter intra-class compactness and larger inter-class separation in the embedding space. Extensive experiments on the IMDB dataset demonstrate that the proposed model achieves competitive performance with an accuracy of 94.67\%, outperforming strong baselines by 1.5--2.5 percentage points. The framework is lightweight, efficient, and readily extensible to other text classification tasks.