Automatic Emotion Modelling in Written Stories

TL;DR

This paper introduces a novel Transformer-based approach for automatically modeling emotional trajectories in written stories by predicting continuous valence and arousal signals, filling a gap in benchmark datasets.

Contribution

It provides the first labeled benchmark dataset with continuous emotion annotations for children's stories and proposes new Transformer-based methods for emotion prediction.

Findings

Achieved CCC of .7338 for valence and .6302 for arousal

Demonstrated the effectiveness of context-aware models

Provided publicly available code and annotations

Abstract

Telling stories is an integral part of human communication which can evoke emotions and influence the affective states of the audience. Automatically modelling emotional trajectories in stories has thus attracted considerable scholarly interest. However, as most existing works have been limited to unsupervised dictionary-based approaches, there is no labelled benchmark for this task. We address this gap by introducing continuous valence and arousal annotations for an existing dataset of children's stories annotated with discrete emotion categories. We collect additional annotations for this data and map the originally categorical labels to the valence and arousal space. Leveraging recent advances in Natural Language Processing, we propose a set of novel Transformer-based methods for predicting valence and arousal signals over the course of written stories. We explore several strategies for fine-tuning a pretrained ELECTRA model and study the benefits of considering a sentence's context when inferring its emotionality. Moreover, we experiment with additional LSTM and Transformer layers. The best configuration achieves a Concordance Correlation Coefficient (CCC) of .7338 for valence and .6302 for arousal on the test set, demonstrating the suitability of our proposed approach. Our code and additional annotations are made available at https://github.com/lc0197/emotion_modelling_stories.