A Causal And-Or Graph Model for Visibility Fluent Reasoning in Tracking Interacting Objects

TL;DR

This paper introduces a causal And-Or graph model to improve human tracking in videos by reasoning about visibility changes due to interactions, enabling more complete trajectory recovery in crowded scenes.

Contribution

It proposes a novel Causal And-Or Graph model that jointly reasons about visibility fluents and tracks humans, addressing challenges in crowded, interactive environments.

Findings

Outperforms existing trackers in complex scenarios

Accurately estimates visibility fluent changes over time

Recovers complete human trajectories in crowded scenes

Abstract

Tracking humans that are interacting with the other subjects or environment remains unsolved in visual tracking, because the visibility of the human of interests in videos is unknown and might vary over time. In particular, it is still difficult for state-of-the-art human trackers to recover complete human trajectories in crowded scenes with frequent human interactions. In this work, we consider the visibility status of a subject as a fluent variable, whose change is mostly attributed to the subject's interaction with the surrounding, e.g., crossing behind another object, entering a building, or getting into a vehicle, etc. We introduce a Causal And-Or Graph (C-AOG) to represent the causal-effect relations between an object's visibility fluent and its activities, and develop a probabilistic graph model to jointly reason the visibility fluent change (e.g., from visible to invisible) and track humans in videos. We formulate this joint task as an iterative search of a feasible causal graph structure that enables fast search algorithm, e.g., dynamic programming method. We apply the proposed method on challenging video sequences to evaluate its capabilities of estimating visibility fluent changes of subjects and tracking subjects of interests over time. Results with comparisons demonstrate that our method outperforms the alternative trackers and can recover complete trajectories of humans in complicated scenarios with frequent human interactions.