KESA: A Knowledge Enhanced Approach For Sentiment Analysis

TL;DR

This paper introduces KESA, a lightweight sentiment knowledge integration method for sentence-level sentiment analysis, using auxiliary tasks that improve semantic understanding and outperform existing models.

Contribution

Proposes two novel sentiment-aware auxiliary tasks, sentiment word cloze and conditional sentiment prediction, to incorporate sentiment knowledge more efficiently into models.

Findings

Outperforms pre-trained models on sentiment analysis tasks

Enhances semantic representations with auxiliary tasks

Complementary to existing knowledge-enhanced models

Abstract

Though some recent works focus on injecting sentiment knowledge into pre-trained language models, they usually design mask and reconstruction tasks in the post-training phase. In this paper, we aim to benefit from sentiment knowledge in a lighter way. To achieve this goal, we study sentence-level sentiment analysis and, correspondingly, propose two sentiment-aware auxiliary tasks named sentiment word cloze and conditional sentiment prediction. The first task learns to select the correct sentiment words within the input, given the overall sentiment polarity as prior knowledge. On the contrary, the second task predicts the overall sentiment polarity given the sentiment polarity of the word as prior knowledge. In addition, two kinds of label combination methods are investigated to unify multiple types of labels in each task. We argue that more information can promote the models to learn more profound semantic representation. We implement it in a straightforward way to verify this hypothesis. The experimental results demonstrate that our approach consistently outperforms pre-trained models and is additive to existing knowledge-enhanced post-trained models. The code and data are released at https://github.com/lshowway/KESA.