Relation3DMOT: Exploiting Deep Affinity for 3D Multi-Object Tracking from View Aggregation

TL;DR

This paper introduces Relation3DMOT, a novel 3D multi-object tracking method that fuses 2D and 3D features and employs a new convolutional operation to improve data association, achieving state-of-the-art results.

Contribution

The paper proposes a joint feature extractor and a new RelationConv operation to better exploit object correlations for improved 3D MOT.

Findings

Achieves state-of-the-art performance on KITTI benchmark.

Effectively fuses 2D and 3D appearance features.

Improves data association accuracy with RelationConv.

Abstract

Autonomous systems need to localize and track surrounding objects in 3D space for safe motion planning. As a result, 3D multi-object tracking (MOT) plays a vital role in autonomous navigation. Most MOT methods use a tracking-by-detection pipeline, which includes object detection and data association processing. However, many approaches detect objects in 2D RGB sequences for tracking, which is lack of reliability when localizing objects in 3D space. Furthermore, it is still challenging to learn discriminative features for temporally-consistent detection in different frames, and the affinity matrix is normally learned from independent object features without considering the feature interaction between detected objects in the different frames. To settle these problems, We firstly employ a joint feature extractor to fuse the 2D and 3D appearance features captured from both 2D RGB images and 3D point clouds respectively, and then propose a novel convolutional operation, named RelationConv, to better exploit the correlation between each pair of objects in the adjacent frames, and learn a deep affinity matrix for further data association. We finally provide extensive evaluation to reveal that our proposed model achieves state-of-the-art performance on KITTI tracking benchmark.