IITK at SemEval-2020 Task 8: Unimodal and Bimodal Sentiment Analysis of Internet Memes

TL;DR

This paper explores unimodal and bimodal sentiment analysis of internet memes using NLP and CV techniques, finding that a simple text-only approach outperforms more complex models and achieves top results in SemEval-2020 Task 8.

Contribution

It demonstrates that a straightforward text-only neural network with Word2vec embeddings can outperform multimodal models in meme sentiment classification.

Findings

Text-only approach outperforms multimodal models

Achieved 63% relative improvement over baseline

Ranked first in SemEval-2020 Task 8 sentiment analysis

Abstract

Social media is abundant in visual and textual information presented together or in isolation. Memes are the most popular form, belonging to the former class. In this paper, we present our approaches for the Memotion Analysis problem as posed in SemEval-2020 Task 8. The goal of this task is to classify memes based on their emotional content and sentiment. We leverage techniques from Natural Language Processing (NLP) and Computer Vision (CV) towards the sentiment classification of internet memes (Subtask A). We consider Bimodal (text and image) as well as Unimodal (text-only) techniques in our study ranging from the Na\"ive Bayes classifier to Transformer-based approaches. Our results show that a text-only approach, a simple Feed Forward Neural Network (FFNN) with Word2vec embeddings as input, performs superior to all the others. We stand first in the Sentiment analysis task with a relative improvement of 63% over the baseline macro-F1 score. Our work is relevant to any task concerned with the combination of different modalities.