Entity-Level Sentiment: More than the Sum of Its Parts

TL;DR

This paper investigates how sentiment towards specific entities in longer texts is expressed and modeled, revealing that aggregate sentence sentiments often do not match overall entity sentiment, highlighting the complexity of entity-level sentiment analysis.

Contribution

It introduces a new dataset with expert annotations for entity-specific sentiment in longer texts and analyzes the discrepancy between sentence-level and overall entity sentiment.

Findings

Only 70% of positive entities are correctly labeled overall

55% of negative entities receive correct overall sentiment

The dataset enables more precise modeling of entity-specific sentiment

Abstract

In sentiment analysis of longer texts, there may be a variety of topics discussed, of entities mentioned, and of sentiments expressed regarding each entity. We find a lack of studies exploring how such texts express their sentiment towards each entity of interest, and how these sentiments can be modelled. In order to better understand how sentiment regarding persons and organizations (each entity in our scope) is expressed in longer texts, we have collected a dataset of expert annotations where the overall sentiment regarding each entity is identified, together with the sentence-level sentiment for these entities separately. We show that the reader's perceived sentiment regarding an entity often differs from an arithmetic aggregation of sentiments at the sentence level. Only 70\% of the positive and 55\% of the negative entities receive a correct overall sentiment label when we aggregate the (human-annotated) sentiment labels for the sentences where the entity is mentioned. Our dataset reveals the complexity of entity-specific sentiment in longer texts, and allows for more precise modelling and evaluation of such sentiment expressions.