TripletTrack: 3D Object Tracking using Triplet Embeddings and LSTM

TL;DR

TripletTrack introduces a 3D object tracking method combining triplet embeddings and LSTM-based motion modeling, improving re-identification and robustness in autonomous driving scenarios with inexpensive sensors.

Contribution

The paper proposes a novel 3D tracking approach that integrates visual appearance embeddings with motion descriptors, enhancing re-identification and occlusion handling.

Findings

Outperforms state-of-the-art on nuScenes dataset

Achieves competitive results on KITTI dataset

Effectively handles occlusions and missed detections

Abstract

3D object tracking is a critical task in autonomous driving systems. It plays an essential role for the system's awareness about the surrounding environment. At the same time there is an increasing interest in algorithms for autonomous cars that solely rely on inexpensive sensors, such as cameras. In this paper we investigate the use of triplet embeddings in combination with motion representations for 3D object tracking. We start from an off-the-shelf 3D object detector, and apply a tracking mechanism where objects are matched by an affinity score computed on local object feature embeddings and motion descriptors. The feature embeddings are trained to include information about the visual appearance and monocular 3D object characteristics, while motion descriptors provide a strong representation of object trajectories. We will show that our approach effectively re-identifies objects, and also behaves reliably and accurately in case of occlusions, missed detections and can detect re-appearance across different field of views. Experimental evaluation shows that our approach outperforms state-of-the-art on nuScenes by a large margin. We also obtain competitive results on KITTI.