Reconstructing 3D Motion Trajectory of Large Swarm of Flying Objects

TL;DR

This paper presents a unified optimization method for reconstructing 3D trajectories of large swarms of flying objects using synchronized cameras, overcoming ambiguities in stereo matching and tracking.

Contribution

It introduces a joint approach that simultaneously solves stereo matching and temporal tracking, improving accuracy in large-scale 3D motion reconstruction.

Findings

Significant performance improvement over existing methods.

Successful reconstruction of hundreds of fruit flies' trajectories.

Facilitates studies of insect collective behavior.

Abstract

This paper addresses the problem of reconstructing the motion trajectories of the individuals in a large collection of flying objects using two temporally synchronized and geometrically calibrated cameras. The 3D trajectory reconstruction problem involves two challenging tasks - stereo matching and temporal tracking. Existing methods separate the two and process them one at a time sequentially, and suffer from frequent irresolvable ambiguities in stereo matching and in tracking. We unify the two tasks, and propose an optimization approach to solving stereo matching and temporal tracking simultaneously. It treats 3D trajectory acquisition problem as selecting appropriate stereo correspondence out of all possible ones for each object via minimizing a cost function. Experiment results show that the proposed method offers significant performance advantage over existing approaches. The proposed method has successfully been applied to reconstruct 3D motion trajectories of hundreds of simultaneously flying fruit flies (Drosophila Melanogaster), which could facilitate the study the insect's collective behavior.