Detecting Offensive Memes with Social Biases in Singapore Context Using Multimodal Large Language Models

TL;DR

This paper develops a multimodal large language model-based system to detect offensive memes in Singapore, addressing cultural and linguistic nuances, achieving over 80% accuracy to assist content moderation.

Contribution

It introduces a large, culturally specific dataset and fine-tunes a multimodal model for offensive meme detection in Singapore's multilingual context.

Findings

Achieved 80.62% accuracy on test set

Proposed a pipeline with OCR, translation, and VLM

Open-sourced dataset, code, and models

Abstract

Traditional online content moderation systems struggle to classify modern multimodal means of communication, such as memes, a highly nuanced and information-dense medium. This task is especially hard in a culturally diverse society like Singapore, where low-resource languages are used and extensive knowledge on local context is needed to interpret online content. We curate a large collection of 112K memes labeled by GPT-4V for fine-tuning a VLM to classify offensive memes in Singapore context. We show the effectiveness of fine-tuned VLMs on our dataset, and propose a pipeline containing OCR, translation and a 7-billion parameter-class VLM. Our solutions reach 80.62% accuracy and 0.8192 AUROC on a held-out test set, and can greatly aid human in moderating online contents. The dataset, code, and model weights have been open-sourced at https://github.com/aliencaocao/vlm-for-memes-aisg.