Uncovering the Limits of Text-based Emotion Detection

TL;DR

This paper evaluates emotion detection from text using large datasets, introduces new models that outperform existing ones, and analyzes differences between writer and reader perceptions of emotions.

Contribution

It presents a benchmark for emotion classification, introduces novel BERT-based models, and provides insights into writer versus reader emotion perception differences.

Findings

Writer-expressed emotions are harder to identify than reader-perceived emotions.

New BERT-based models outperform previous baselines on GoEmotions.

A public web interface is provided for further research exploration.

Abstract

Identifying emotions from text is crucial for a variety of real world tasks. We consider the two largest now-available corpora for emotion classification: GoEmotions, with 58k messages labelled by readers, and Vent, with 33M writer-labelled messages. We design a benchmark and evaluate several feature spaces and learning algorithms, including two simple yet novel models on top of BERT that outperform previous strong baselines on GoEmotions. Through an experiment with human participants, we also analyze the differences between how writers express emotions and how readers perceive them. Our results suggest that emotions expressed by writers are harder to identify than emotions that readers perceive. We share a public web interface for researchers to explore our models.