MemeCMD: An Automatically Generated Chinese Multi-turn Dialogue Dataset with Contextually Retrieved Memes

TL;DR

MemeCMD is a large-scale Chinese multi-turn dialogue dataset that incorporates contextually relevant memes, created through automatic generation and retrieval methods to enhance multimodal conversational AI.

Contribution

The paper introduces MemeCMD, a novel automatically generated dataset combining dialogues with contextually retrieved memes for Chinese multi-turn conversations.

Findings

Effective retrieval framework for relevant memes

Diverse and contextually appropriate meme-incorporated dialogues

Scalable, privacy-preserving dataset for multimodal AI

Abstract

Memes are widely used in online social interactions, providing vivid, intuitive, and often humorous means to express intentions and emotions. Existing dialogue datasets are predominantly limited to either manually annotated or pure-text conversations, lacking the expressiveness and contextual nuance that multimodal interactions provide.To address these challenges, we introduce MemeCMD, an automatically generated Chinese Multi-turn Dialogue dataset with contextually retrieved memes. Our dataset combines a large-scale, MLLM-annotated meme library with dialogues auto-generated by dual agents across diverse scenarios. We introduce a retrieval framework and adaptive threshold to ensure contextually relevant, naturally spaced meme usage. Experiments demonstrate the effectiveness of our approach in generating contextually appropriate and diverse meme-incorporated dialogues, offering a scalable and privacy-preserving resource for advancing multimodal conversational AI.