Author's Sentiment Prediction

TL;DR

This paper introduces PerSenT, a challenging new dataset for author sentiment analysis in news articles, highlighting the difficulty of fine-tuning existing models for this nuanced task.

Contribution

The paper presents PerSenT, a novel dataset with paragraph-level annotations for author sentiment, and provides benchmarks showing the limitations of current models.

Findings

Fine-tuning BERT alone is insufficient for this task.

Paragraph-level aggregation methods are ineffective.

The dataset reveals specific challenges in entity sentiment analysis.

Abstract

We introduce PerSenT, a dataset of crowd-sourced annotations of the sentiment expressed by the authors towards the main entities in news articles. The dataset also includes paragraph-level sentiment annotations to provide more fine-grained supervision for the task. Our benchmarks of multiple strong baselines show that this is a difficult classification task. The results also suggest that simply fine-tuning document-level representations from BERT isn't adequate for this task. Making paragraph-level decisions and aggregating them over the entire document is also ineffective. We present empirical and qualitative analyses that illustrate the specific challenges posed by this dataset. We release this dataset with 5.3k documents and 38k paragraphs covering 3.2k unique entities as a challenge in entity sentiment analysis.