OSPC: Detecting Harmful Memes with Large Language Model as a Catalyst

TL;DR

This paper introduces a comprehensive system combining image captioning, OCR, and large language models to detect harmful memes across multiple languages, achieving state-of-the-art performance in a Singaporean context.

Contribution

The study presents a novel multi-modal framework integrating various AI models and fine-tuning with GPT-4V data for effective harmful meme detection in multilingual settings.

Findings

Achieved top-1 at AI Singapore's Online Safety Prize Challenge

Outperformed previous benchmarks like FLAVA and VisualBERT

System effectively detects harmful content in four languages

Abstract

Memes, which rapidly disseminate personal opinions and positions across the internet, also pose significant challenges in propagating social bias and prejudice. This study presents a novel approach to detecting harmful memes, particularly within the multicultural and multilingual context of Singapore. Our methodology integrates image captioning, Optical Character Recognition (OCR), and Large Language Model (LLM) analysis to comprehensively understand and classify harmful memes. Utilizing the BLIP model for image captioning, PP-OCR and TrOCR for text recognition across multiple languages, and the Qwen LLM for nuanced language understanding, our system is capable of identifying harmful content in memes created in English, Chinese, Malay, and Tamil. To enhance the system's performance, we fine-tuned our approach by leveraging additional data labeled using GPT-4V, aiming to distill the understanding capability of GPT-4V for harmful memes to our system. Our framework achieves top-1 at the public leaderboard of the Online Safety Prize Challenge hosted by AI Singapore, with the AUROC as 0.7749 and accuracy as 0.7087, significantly ahead of the other teams. Notably, our approach outperforms previous benchmarks, with FLAVA achieving an AUROC of 0.5695 and VisualBERT an AUROC of 0.5561.