Incorporating Dynamic Semantics into Pre-Trained Language Model for Aspect-based Sentiment Analysis

TL;DR

This paper introduces DR-BERT, a novel approach that dynamically re-weights BERT's semantics to improve aspect-based sentiment analysis, achieving better interpretability and performance on benchmark datasets.

Contribution

The paper proposes a new dynamic re-weighting mechanism integrated into BERT for aspect-based sentiment analysis, enhancing semantic focus and interpretability.

Findings

Improved sentiment classification accuracy on benchmark datasets.

Enhanced interpretability through dynamic word importance re-weighting.

Effective modeling of semantic shifts in aspect-based sentiment analysis.

Abstract

Aspect-based sentiment analysis (ABSA) predicts sentiment polarity towards a specific aspect in the given sentence. While pre-trained language models such as BERT have achieved great success, incorporating dynamic semantic changes into ABSA remains challenging. To this end, in this paper, we propose to address this problem by Dynamic Re-weighting BERT (DR-BERT), a novel method designed to learn dynamic aspect-oriented semantics for ABSA. Specifically, we first take the Stack-BERT layers as a primary encoder to grasp the overall semantic of the sentence and then fine-tune it by incorporating a lightweight Dynamic Re-weighting Adapter (DRA). Note that the DRA can pay close attention to a small region of the sentences at each step and re-weigh the vitally important words for better aspect-aware sentiment understanding. Finally, experimental results on three benchmark datasets demonstrate the effectiveness and the rationality of our proposed model and provide good interpretable insights for future semantic modeling.