SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!

TL;DR

This paper introduces a multimodal meme analysis task, Memotion, which involves classifying sentiment, emotion type, and intensity of Internet memes using a dataset of approximately 10,000 annotated memes, highlighting the need for hybrid visual-lingual approaches.

Contribution

It presents a new multimodal meme analysis benchmark with annotated data and defines three subtasks for sentiment, emotion type, and intensity classification.

Findings

Achieved F1 scores of 0.35, 0.51, and 0.32 on the three subtasks.

Highlights the importance of multimodal approaches for meme emotion analysis.

Provides a new dataset for future research in meme understanding.

Abstract

Information on social media comprises of various modalities such as textual, visual and audio. NLP and Computer Vision communities often leverage only one prominent modality in isolation to study social media. However, the computational processing of Internet memes needs a hybrid approach. The growing ubiquity of Internet memes on social media platforms such as Facebook, Instagram, and Twiter further suggests that we can not ignore such multimodal content anymore. To the best of our knowledge, there is not much attention towards meme emotion analysis. The objective of this proposal is to bring the attention of the research community towards the automatic processing of Internet memes. The task Memotion analysis released approx 10K annotated memes, with human-annotated labels namely sentiment (positive, negative, neutral), type of emotion (sarcastic, funny, offensive, motivation) and their corresponding intensity. The challenge consisted of three subtasks: sentiment (positive, negative, and neutral) analysis of memes, overall emotion (humour, sarcasm, offensive, and motivational) classification of memes, and classifying intensity of meme emotion. The best performances achieved were F1 (macro average) scores of 0.35, 0.51 and 0.32, respectively for each of the three subtasks.