Finding the Trigger: Causal Abductive Reasoning on Video Events

TL;DR

This paper defines the new task of causal abductive reasoning in videos, introduces datasets and a neural framework to identify causal event chains, advancing research in video understanding and causal inference.

Contribution

It introduces the CARVE task, creates benchmark datasets with a novel synthesis approach, and proposes the CERN model for causal reasoning in videos.

Findings

Event relational learning is crucial for causal reasoning.

CERN effectively identifies causal triggers in videos.

Datasets enable future research in video causal inference.

Abstract

This paper introduces a new problem, Causal Abductive Reasoning on Video Events (CARVE), which involves identifying causal relationships between events in a video and generating hypotheses about causal chains that account for the occurrence of a target event. To facilitate research in this direction, we create two new benchmark datasets with both synthetic and realistic videos, accompanied by trigger-target labels generated through a novel counterfactual synthesis approach. To explore the challenge of solving CARVE, we present a Causal Event Relation Network (CERN) that examines the relationships between video events in temporal and semantic spaces to efficiently determine the root-cause trigger events. Through extensive experiments, we demonstrate the critical roles of event relational representation learning and interaction modeling in solving video causal reasoning challenges. The introduction of the CARVE task, along with the accompanying datasets and the CERN framework, will advance future research on video causal reasoning and significantly facilitate various applications, including video surveillance, root-cause analysis and movie content management.