GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art

TL;DR

GODBench is a new benchmark for evaluating multimodal large language models' ability to create creative video comments, addressing current limitations in understanding and generating humor, satire, and emotional content.

Contribution

The paper introduces GODBench, a comprehensive multimodal benchmark, and Ripple of Thought, a reasoning framework to enhance creativity in video comment generation by MLLMs.

Findings

Existing MLLMs struggle with creative video comments.

Ripple of Thought improves creative reasoning in MLLMs.

GODBench enables systematic evaluation of multimodal creativity.

Abstract

Video Comment Art enhances user engagement by providing creative content that conveys humor, satire, or emotional resonance, requiring a nuanced and comprehensive grasp of cultural and contextual subtleties. Although Multimodal Large Language Models (MLLMs) and Chain-of-Thought (CoT) have demonstrated strong reasoning abilities in STEM tasks (e.g. mathematics and coding), they still struggle to generate creative expressions such as resonant jokes and insightful satire. Moreover, existing benchmarks are constrained by their limited modalities and insufficient categories, hindering the exploration of comprehensive creativity in video-based Comment Art creation. To address these limitations, we introduce GODBench, a novel benchmark that integrates video and text modalities to systematically evaluate MLLMs' abilities to compose Comment Art. Furthermore, inspired by the propagation patterns of waves in physics, we propose Ripple of Thought (RoT), a multi-step reasoning framework designed to enhance the creativity of MLLMs. Extensive experiments reveal that existing MLLMs and CoT methods still face significant challenges in understanding and generating creative video comments. In contrast, RoT provides an effective approach to improve creative composing, highlighting its potential to drive meaningful advancements in MLLM-based creativity. GODBench is publicly available at https://github.com/stan-lei/GODBench-ACL2025.