gundapusunil at SemEval-2020 Task 8: Multimodal Memotion Analysis

TL;DR

This paper presents a multimodal deep learning system for classifying Internet memes by sentiment, humor type, and intensity, combining computer vision and NLP techniques to go beyond traditional sentiment analysis.

Contribution

It introduces a novel multimodal approach using CNN and LSTM for detailed meme classification, including humor and effect quantification.

Findings

Outperformed baseline scores in meme classification tasks

Effectively combined visual and textual data for nuanced analysis

Achieved high accuracy in sentiment and humor type detection

Abstract

Recent technological advancements in the Internet and Social media usage have resulted in the evolution of faster and efficient platforms of communication. These platforms include visual, textual and speech mediums and have brought a unique social phenomenon called Internet memes. Internet memes are in the form of images with witty, catchy, or sarcastic text descriptions. In this paper, we present a multi-modal sentiment analysis system using deep neural networks combining Computer Vision and Natural Language Processing. Our aim is different than the normal sentiment analysis goal of predicting whether a text expresses positive or negative sentiment; instead, we aim to classify the Internet meme as a positive, negative, or neutral, identify the type of humor expressed and quantify the extent to which a particular effect is being expressed. Our system has been developed using CNN and LSTM and outperformed the baseline score.