PrismAgent: Illuminating Harm in Memes via a Zero-Shot Interpretable Multi-Agent Framework

TL;DR

PrismAgent is a zero-shot, interpretable multi-agent framework that detects harmful memes by simulating a criminal investigation process, significantly outperforming existing methods without relying on annotated training data.

Contribution

It introduces a novel multi-agent, interpretability-focused approach for harmful meme detection that operates effectively in zero-shot settings, reducing reliance on annotated datasets.

Findings

Outperforms existing zero-shot detection methods on three datasets.

Provides explicit, interpretable reasoning steps for each detection.

Demonstrates effectiveness in identifying harmful content without training data.

Abstract

The rapid spread of memes makes harmful content detection increasingly crucial, as effective identification can curb the circulation of misinformation. However, existing methods rely heavily on high-volume annotated data, which leads to substantial training costs and limited generalization. To address these challenges, we propose PrismAgent, a zero-shot, multi-agent, interpretable framework. PrismAgent conceptualizes this task as a criminal case investigation, employing four specialized agents responsible for the analysis, investigation, prosecution, and judgment stages within a structured collaborative workflow. In the first stage, the analyst agent paraphrases each meme under benevolent and malicious assumptions to probe its underlying intent. The investigator agent then retrieves supporting evidence from an unannotated dataset and constructs contextual interpretations for the meme and its variants. Next, the prosecutor agent performs three independent preliminary judgments by pairing the original meme with each of the three interpretations. Finally, the judge agent deliberates across all evidence to render a final verdict. Moreover, PrismAgent's explicit multi-stage reasoning chain makes the model inherently interpretable, as every intermediate step is explicitly explained rather than only producing a final detection result. Extensive experiments on three public datasets show that PrismAgent significantly outperforms existing zero-shot detection methods.