Estimating Dynamic Flow Features in Groups of Tracked Objects

TL;DR

This paper introduces a method to analyze complex motion patterns in groups of tracked objects within image sequences by estimating flow features using deep learning and gradient regression, enabling insights into underlying dynamics.

Contribution

It extends gradient-based dynamical analysis to real-world, feature-rich image data with imperfect tracers using deep vision networks and Lagrangian gradient regression.

Findings

Successfully identifies regions of rotation and transport barriers.

Enables analysis of multiple object classes in a single sequence.

Applicable to complex, real-world image data.

Abstract

Interpreting motion captured in image sequences is crucial for a wide range of computer vision applications. Typical estimation approaches include optical flow (OF), which approximates the apparent motion instantaneously in a scene, and multiple object tracking (MOT), which tracks the motion of subjects over time. Often, the motion of objects in a scene is governed by some underlying dynamical system which could be inferred by analyzing the motion of groups of objects. Standard motion analyses, however, are not designed to intuit flow dynamics from trajectory data, making such measurements difficult in practice. The goal of this work is to extend gradient-based dynamical systems analyses to real-world applications characterized by complex, feature-rich image sequences with imperfect tracers. The tracer trajectories are tracked using deep vision networks and gradients are approximated using Lagrangian gradient regression (LGR), a tool designed to estimate spatial gradients from sparse data. From gradients, dynamical features such as regions of coherent rotation and transport barriers are identified. The proposed approach is affordably implemented and enables advanced studies including the motion analysis of two distinct object classes in a single image sequence. Two examples of the method are presented on data sets for which standard gradient-based analyses do not apply.