Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm

TL;DR

This paper demonstrates that leveraging a vast dataset of 1246 million tweets with diverse emojis as noisy labels enables training a single model that achieves state-of-the-art results across multiple sentiment, emotion, and sarcasm detection benchmarks.

Contribution

The study extends distant supervision in NLP by using a large, diverse set of emoji labels, resulting in richer representations and improved performance across various emotion-related tasks.

Findings

State-of-the-art performance on 8 benchmark datasets

Diverse emoji labels improve model performance

Large-scale emoji dataset enhances NLP representations

Abstract

NLP tasks are often limited by scarcity of manually annotated data. In social media sentiment analysis and related tasks, researchers have therefore used binarized emoticons and specific hashtags as forms of distant supervision. Our paper shows that by extending the distant supervision to a more diverse set of noisy labels, the models can learn richer representations. Through emoji prediction on a dataset of 1246 million tweets containing one of 64 common emojis we obtain state-of-the-art performance on 8 benchmark datasets within sentiment, emotion and sarcasm detection using a single pretrained model. Our analyses confirm that the diversity of our emotional labels yield a performance improvement over previous distant supervision approaches.