Understanding Textual Emotion Through Emoji Prediction

TL;DR

This study evaluates various deep learning models for emoji prediction from short texts, highlighting BERT's overall superiority and CNN's effectiveness on rare emojis, emphasizing architecture choice and tuning for better sentiment understanding.

Contribution

It compares four deep learning architectures for emoji prediction, demonstrating the impact of model selection and hyperparameter tuning on performance.

Findings

BERT achieves the highest overall accuracy.

CNN performs best on rare emoji classes.

Model choice significantly affects sentiment-aware emoji prediction.

Abstract

This project explores emoji prediction from short text sequences using four deep learning architectures: a feed-forward network, CNN, transformer, and BERT. Using the TweetEval dataset, we address class imbalance through focal loss and regularization techniques. Results show BERT achieves the highest overall performance due to its pre-training advantage, while CNN demonstrates superior efficacy on rare emoji classes. This research shows the importance of architecture selection and hyperparameter tuning for sentiment-aware emoji prediction, contributing to improved human-computer interaction.