Towards a tracking algorithm based on the clustering of spatio-temporal clouds of points

TL;DR

This paper introduces a 3D+1 clustering-based tracking algorithm that efficiently handles optical occlusions and physical proximity in dense, noisy biological data, improving over existing NP-hard set-cover approaches.

Contribution

A novel 3D+1 clustering method that simplifies tracking by reducing complexity and effectively managing occlusions and proximity issues in noisy biological datasets.

Findings

Reduces tracking complexity from NP-hard to P using connected components.

Effectively handles optical occlusions with simple clustering.

Addresses physical proximity challenges with spectral clustering.

Abstract

The interest in 3D dynamical tracking is growing in fields such as robotics, biology and fluid dynamics. Recently, a major source of progress in 3D tracking has been the study of collective behaviour in biological systems, where the trajectories of individual animals moving within large and dense groups need to be reconstructed to understand the behavioural interaction rules. Experimental data in this field are generally noisy and at low spatial resolution, so that individuals appear as small featureless objects and trajectories must be retrieved by making use of epipolar information only. Moreover, optical occlusions often occur: in a multi-camera system one or more objects become indistinguishable in one view, potentially jeopardizing the conservation of identity over long-time trajectories. The most advanced 3D tracking algorithms overcome optical occlusions making use of set-cover techniques, which however have to solve NP-hard optimization problems. Moreover, current methods are not able to cope with occlusions arising from actual physical proximity of objects in 3D space. Here, we present a new method designed to work directly in 3D space and time, creating (3D+1) clouds of points representing the full spatio-temporal evolution of the moving targets. We can then use a simple connected components labeling routine, which is linear in time, to solve optical occlusions, hence lowering from NP to P the complexity of the problem. Finally, we use normalized cut spectral clustering to tackle 3D physical proximity.