Multimodal Analysis of memes for sentiment extraction

TL;DR

This paper introduces three transformer-based methods for multimodal meme sentiment analysis, achieving notable classification performance on the Memotion dataset across various sentiment categories.

Contribution

It presents novel transformer-based techniques specifically designed for multimodal meme sentiment classification, advancing the state of the art in this domain.

Findings

Best algorithm achieved macro F1 of 0.633 for humour

Achieved macro F1 of 0.55 for motivation

Achieved macro F1 of 0.61 for sarcasm

Abstract

Memes are one of the most ubiquitous forms of social media communication. The study and processing of memes, which are intrinsically multimedia, is a popular topic right now. The study presented in this research is based on the Memotion dataset, which involves categorising memes based on irony, comedy, motivation, and overall-sentiment. Three separate innovative transformer-based techniques have been developed, and their outcomes have been thoroughly reviewed.The best algorithm achieved a macro F1 score of 0.633 for humour classification, 0.55 for motivation classification, 0.61 for sarcasm classification, and 0.575 for overall sentiment of the meme out of all our techniques.