RiskCueBench: Benchmarking Anticipatory Reasoning from Early Risk Cues in Video-Language Models

TL;DR

RiskCueBench is a new benchmark designed to evaluate how well video-language models can anticipate risky events from early visual cues, highlighting current limitations in early risk detection.

Contribution

The paper introduces RiskCueBench, a novel benchmark dataset focusing on early risk cue detection in videos, addressing limitations of existing datasets that include full event videos.

Findings

Current models struggle to interpret early risk signals

Significant gap between human and model performance in early risk prediction

Challenges identified for deploying real-time risk anticipation systems

Abstract

With the rapid growth of video centered social media, the ability to anticipate risky events from visual data is a promising direction for ensuring public safety and preventing real world accidents. Prior work has extensively studied supervised video risk assessment across domains such as driving, protests, and natural disasters. However, many existing datasets provide models with access to the full video sequence, including the accident itself, which substantially reduces the difficulty of the task. To better reflect real world conditions, we introduce a new video understanding benchmark RiskCueBench in which videos are carefully annotated to identify a risk signal clip, defined as the earliest moment that indicates a potential safety concern. Experimental results reveal a significant gap in current systems ability to interpret evolving situations and anticipate future risky events from early visual signals, highlighting important challenges for deploying video risk prediction models in practice.