Attention-Based Neural Networks for Sentiment Attitude Extraction using Distant Supervision

TL;DR

This paper investigates attention-based neural network encoders for sentiment attitude extraction in Russian texts, demonstrating improved classification performance with attention mechanisms and analyzing attention weight distributions.

Contribution

It introduces and evaluates attention-based context encoders for sentiment attitude extraction, utilizing Russian corpora and showing performance gains over non-attention models.

Findings

Attention mechanisms improve F1 scores by 3-10%.

Three-class models outperform two-class models.

Attention weight distribution analysis reveals term-type dependencies.

Abstract

In the sentiment attitude extraction task, the aim is to identify <<attitudes>> -- sentiment relations between entities mentioned in text. In this paper, we provide a study on attention-based context encoders in the sentiment attitude extraction task. For this task, we adapt attentive context encoders of two types: (1) feature-based; (2) self-based. In our study, we utilize the corpus of Russian analytical texts RuSentRel and automatically constructed news collection RuAttitudes for enriching the training set. We consider the problem of attitude extraction as two-class (positive, negative) and three-class (positive, negative, neutral) classification tasks for whole documents. Our experiments with the RuSentRel corpus show that the three-class classification models, which employ the RuAttitudes corpus for training, result in 10% increase and extra 3% by F1, when model architectures include the attention mechanism. We also provide the analysis of attention weight distributions in dependence on the term type.