From Perception to Punchline: Empowering VLM with the Art of In-the-wild Meme

TL;DR

This paper introduces HUMOR, a framework that enhances vision-language models to generate humorous memes by hierarchical reasoning and aligning with human preferences, resulting in more diverse and higher-quality meme creation.

Contribution

HUMOR employs a multi-path Chain-of-Thought reasoning and a group-wise reward model to improve humor quality and subjective preference alignment in multimodal meme generation.

Findings

HUMOR achieves superior reasoning diversity in meme generation.

The reward model ensures consistent alignment with human preferences.

HUMOR improves overall meme quality across various models.

Abstract

Generating humorous memes is a challenging multimodal task that moves beyond direct image-to-caption supervision. It requires a nuanced reasoning over visual content, contextual cues, and subjective humor. To bridge this gap between visual perception and humorous punchline creation, we propose HUMOR}, a novel framework that guides VLMs through hierarchical reasoning and aligns them with group-wise human preferences. First, HUMOR employs a hierarchical, multi-path Chain-of-Thought (CoT): the model begins by identifying a template-level intent, then explores diverse reasoning paths under different contexts, and finally anchors onto a high-quality, context-specific path. This CoT supervision, which traces back from ground-truth captions, enhances reasoning diversity. We further analyze that this multi-path exploration with anchoring maintains a high expected humor quality, under the practical condition that high-quality paths retain significant probability mass. Second, to capture subjective humor, we train a pairwise reward model that operates within groups of memes sharing the same template. Following established theory, this approach ensures a consistent and robust proxy for human preference, even with subjective and noisy labels. The reward model then enables a group-wise reinforcement learning optimization, guaranteeing providing a theoretical guarantee for monotonic improvement within the trust region. Extensive experiments show that HUMOR empowers various VLMs with superior reasoning diversity, more reliable preference alignment, and higher overall meme quality. Beyond memes, our work presents a general training paradigm for open-ended, human-aligned multimodal generation, where success is guided by comparative judgment within coherent output group.