RubCSG at SemEval-2022 Task 5: Ensemble learning for identifying misogynous MEMEs

TL;DR

This paper introduces an ensemble learning system combining uni-modal and bi-modal models to identify and classify misogynous memes, achieving top-11 and top-10 rankings in SemEval-2022 Task 5.

Contribution

It proposes a novel model fusion network and ensemble approach specifically designed for misogyny detection in memes, improving performance over existing methods.

Findings

Achieved 0.755 macro F1-score in sub-task A

Achieved 0.709 weighted F1-score in sub-task B

Outperformed many competing systems in the challenge

Abstract

This work presents an ensemble system based on various uni-modal and bi-modal model architectures developed for the SemEval 2022 Task 5: MAMI-Multimedia Automatic Misogyny Identification. The challenge organizers provide an English meme dataset to develop and train systems for identifying and classifying misogynous memes. More precisely, the competition is separated into two sub-tasks: sub-task A asks for a binary decision as to whether a meme expresses misogyny, while sub-task B is to classify misogynous memes into the potentially overlapping sub-categories of stereotype, shaming, objectification, and violence. For our submission, we implement a new model fusion network and employ an ensemble learning approach for better performance. With this structure, we achieve a 0.755 macroaverage F1-score (11th) in sub-task A and a 0.709 weighted-average F1-score (10th) in sub-task B.