VideoASMR-Bench: Can AI-Generated ASMR Videos Fool VLMs and Humans?

TL;DR

VideoASMR-Bench is a new benchmark for evaluating AI-generated ASMR videos' realism and detection, revealing current models' limitations in discerning synthetic from real videos and vice versa.

Contribution

The paper introduces VideoASMR-Bench, a diverse dataset and an adversarial evaluation framework for assessing AI-generated ASMR videos and detection capabilities.

Findings

State-of-the-art VLMs struggle to detect AI-generated ASMR videos.

Current video generation models can produce convincing ASMR videos.

Humans still outperform models in identifying fake ASMR videos.

Abstract

With AI-generated videos increasingly indistinguishable from reality, current benchmarks primarily focus on broad semantic alignment and basic physical consistency, offering limited discriminative power for evaluating them. To address this, we introduce VideoASMR-Bench, a benchmark based on Autonomous Sensory Meridian Response (ASMR) videos that emphasizes fine-grained audio-visual perception and sensory immersion. This benchmark aims to answer two key questions: (i) Are today's video understanding models (VLMs) sensitive enough to detect AI-generated ASMR videos by recognizing minor visual, physical, or auditory artifacts? (ii) Can today's video generation models (VGMs) produce convincing ASMR videos with immersive experiences? This benchmark comprises a diverse set of 1,500 high-quality real ASMR videos curated from social media, alongside 2,235 synthetic counterparts generated by nine VGMs. Additionally, we open-source an extensible suite of prompts and reference images, enabling the benchmark to scale dynamically with future video models. Moreover, we design an automatic understanding-generation evaluation framework between VGMs and VLMs, where VGMs aim to produce realistic fake videos to fool the VLMs, while the VLMs seek to detect them, forming an adversarial game between the two parties. Our evaluation on VideoASMR-Bench reveals that even state-of-the-art VLMs, such as Gemini-3-Pro, fail to reliably detect AI-generated ASMR videos. Meanwhile, current frontier video generation models can produce ASMR videos that are difficult for VLMs to distinguish from real ones, while humans can still identify them relatively easily.