GenVideoLens: Where LVLMs Fall Short in AI-Generated Video Detection?

TL;DR

GenVideoLens is a detailed benchmark that evaluates LVLMs' ability to detect AI-generated videos across multiple authenticity dimensions, revealing strengths and weaknesses in current models.

Contribution

The paper introduces GenVideoLens, a fine-grained benchmark with expert annotations across 15 dimensions, enabling detailed evaluation of LVLMs in AI-generated video detection.

Findings

LVLMs perform well on perceptual cues but struggle with optical, physical, and temporal cues.

Model performance varies widely across dimensions, with smaller models sometimes outperforming larger ones.

Current LVLMs have limited use of temporal information in video detection.

Abstract

In recent years, AI-generated videos have become increasingly realistic and sophisticated. Meanwhile, Large Vision-Language Models (LVLMs) have shown strong potential for detecting such content. However, existing evaluation protocols largely treat the task as a binary classification problem and rely on coarse-grained metrics such as overall accuracy, providing limited insight into where LVLMs succeed or fail. To address this limitation, we introduce GenVideoLens, a fine-grained benchmark that enables dimension-wise evaluation of LVLM capabilities in AI-generated video detection. The benchmark contains 400 highly deceptive AI-generated videos and 100 real videos, annotated by experts across 15 authenticity dimensions covering perceptual, optical, physical, and temporal cues. We evaluate eleven representative LVLMs on this benchmark. Our analysis reveals a pronounced dimensional imbalance. While LVLMs perform relatively well on perceptual cues, they struggle with optical consistency, physical interactions, and temporal-causal reasoning. Model performance also varies substantially across dimensions, with smaller open-source models sometimes outperforming stronger proprietary models on specific authenticity cues. Temporal perturbation experiments further show that current LVLMs make limited use of temporal information. Overall, GenVideoLens provides diagnostic insights into LVLM behavior, revealing key capability gaps and offering guidance for improving future AI-generated video detection systems.