3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation

TL;DR

3D-FCT introduces a real-time Siamese network that leverages temporal correlation features for simultaneous 3D object detection and tracking in LiDAR data, improving accuracy over existing methods.

Contribution

It presents a novel multi-task Siamese architecture that combines detection and tracking using correlation features, enabling real-time performance and enhanced accuracy.

Findings

Achieves 5.57% higher mAP on KITTI dataset

Utilizes correlation features for real-time detection

Extends multi-task learning to include tracking regression

Abstract

3D object detection using LiDAR data remains a key task for applications like autonomous driving and robotics. Unlike in the case of 2D images, LiDAR data is almost always collected over a period of time. However, most work in this area has focused on performing detection independent of the temporal domain. In this paper we present 3D-FCT, a Siamese network architecture that utilizes temporal information to simultaneously perform the related tasks of 3D object detection and tracking. The network is trained to predict the movement of an object based on the correlation features of extracted keypoints across time. Calculating correlation across keypoints only allows for real-time object detection. We further extend the multi-task objective to include a tracking regression loss. Finally, we produce high accuracy detections by linking short-term object tracklets into long term tracks based on the predicted tracks. Our proposed method is evaluated on the KITTI tracking dataset where it is shown to provide an improvement of 5.57% mAP over a state-of-the-art approach.