Extending Multi-Object Tracking systems to better exploit appearance and 3D information

TL;DR

This paper proposes a real-time multi-object tracking system that combines Siamese networks and RNNs to utilize appearance and motion cues, incorporating 3D spatial constraints for improved accuracy in applications like autonomous driving.

Contribution

It introduces a novel joint system integrating Siamese networks and RNNs for multi-object tracking, with heuristics for leveraging 3D information in Bird's Eye View space.

Findings

Effective combination of appearance and motion models for tracking.

Incorporation of 3D spatial constraints improves tracking accuracy.

Real-time performance demonstrated in relevant scenarios.

Abstract

Tracking multiple objects in real time is essential for a variety of real-world applications, with self-driving industry being at the foremost. This work involves exploiting temporally varying appearance and motion information for tracking. Siamese networks have recently become highly successful at appearance based single object tracking and Recurrent Neural Networks have started dominating both motion and appearance based tracking. Our work focuses on combining Siamese networks and RNNs to exploit appearance and motion information respectively to build a joint system capable of real time multi-object tracking. We further explore heuristics based constraints for tracking in the Birds Eye View Space for efficiently exploiting 3D information as a constrained optimization problem for track prediction.