Towards a science of human stories: using sentiment analysis and emotional arcs to understand the building blocks of complex social systems

TL;DR

This paper explores how sentiment analysis and emotional trajectories in stories and social data can reveal fundamental patterns in complex social systems, using methods tested on literature and social media datasets.

Contribution

It introduces a set of six core emotional trajectories in stories and evaluates the robustness of dictionary-based sentiment methods across different text lengths.

Findings

Six core emotional arcs identified in stories

Dictionary methods are robust for longer texts but less so for sentences

Social media data can reveal insights into public health and social movements

Abstract

Given the growing assortment of sentiment measuring instruments, it is imperative to understand which aspects of sentiment dictionaries contribute to both their classification accuracy and their ability to provide richer understanding of texts. Here, we perform detailed, quantitative tests and qualitative assessments of 6 dictionary-based methods applied, and briefly examine a further 20 methods. We show that while inappropriate for sentences, dictionary-based methods are generally robust in their classification accuracy for longer texts. Stories often following distinct emotional trajectories, forming patterns that are meaningful to us. By classifying the emotional arcs for a filtered subset of 4,803 stories from Project Gutenberg's fiction collection, we find a set of six core trajectories which form the building blocks of complex narratives. Of profound scientific interest will be the degree to which we can eventually understand the full landscape of human stories, and data driven approaches will play a crucial role. Finally, we utilize web-scale data from Twitter to study the limits of what social data can tell us about public health, mental illness, discourse around the protest movement of #BlackLivesMatter, discourse around climate change, and hidden networks. We conclude with a review of published works in complex systems that separately analyze charitable donations, the happiness of words in 10 languages, 100 years of daily temperature data across the United States, and Australian Rules Football games.