DSC IIT-ISM at SemEval-2020 Task 8: Bi-Fusion Techniques for Deep Meme Emotion Analysis

TL;DR

This paper introduces bimodal fusion techniques for deep meme emotion analysis, improving sentiment and humor classification accuracy in the SemEval-2020 Memotion task.

Contribution

It presents a novel bimodal fusion approach that leverages inter-modal dependency for better meme sentiment and humor analysis.

Findings

Best system achieved macro F1 scores of 0.357 on sentiment

Achieved 0.510 on humor classification

Improved baseline performance across tasks

Abstract

Memes have become an ubiquitous social media entity and the processing and analysis of suchmultimodal data is currently an active area of research. This paper presents our work on theMemotion Analysis shared task of SemEval 2020, which involves the sentiment and humoranalysis of memes. We propose a system which uses different bimodal fusion techniques toleverage the inter-modal dependency for sentiment and humor classification tasks. Out of all ourexperiments, the best system improved the baseline with macro F1 scores of 0.357 on SentimentClassification (Task A), 0.510 on Humor Classification (Task B) and 0.312 on Scales of SemanticClasses (Task C).