Studying Attention Models in Sentiment Attitude Extraction Task

TL;DR

This paper investigates the effectiveness of attention-based context encoders in sentiment attitude extraction, demonstrating their superiority over non-attention models with a 1.5-5.9% F1 improvement on Russian texts.

Contribution

The study adapts and evaluates feature-based and self-based attentive encoders for sentiment attitude extraction, providing insights into their performance and attention weight distributions.

Findings

Attention models outperform non-attention models by 1.5-5.9% F1.

Attention weight distributions vary with term types.

Attention encoders improve sentiment attitude extraction accuracy.

Abstract

In the sentiment attitude extraction task, the aim is to identify <<attitudes>> -- sentiment relations between entities mentioned in text. In this paper, we provide a study on attention-based context encoders in the sentiment attitude extraction task. For this task, we adapt attentive context encoders of two types: (i) feature-based; (ii) self-based. Our experiments with a corpus of Russian analytical texts RuSentRel illustrate that the models trained with attentive encoders outperform ones that were trained without them and achieve 1.5-5.9% increase by F1. We also provide the analysis of attention weight distributions in dependence on the term type.