Future Does Matter: Boosting 3D Object Detection with Temporal Motion Estimation in Point Cloud Sequences

TL;DR

This paper introduces LiSTM, a novel LiDAR 3D object detection framework that leverages temporal motion estimation and feature aggregation to improve detection accuracy in autonomous driving scenarios.

Contribution

The work proposes a new framework incorporating motion-guided feature aggregation and dual correlation weighting to enhance spatial-temporal feature learning in 3D detection.

Findings

Achieves superior detection performance on Waymo and nuScenes datasets.

Effectively models object trajectories using motion-based heatmaps.

Enhances spatial-temporal feature fusion with novel modules.

Abstract

Accurate and robust LiDAR 3D object detection is essential for comprehensive scene understanding in autonomous driving. Despite its importance, LiDAR detection performance is limited by inherent constraints of point cloud data, particularly under conditions of extended distances and occlusions. Recently, temporal aggregation has been proven to significantly enhance detection accuracy by fusing multi-frame viewpoint information and enriching the spatial representation of objects. In this work, we introduce a novel LiDAR 3D object detection framework, namely LiSTM, to facilitate spatial-temporal feature learning with cross-frame motion forecasting information. We aim to improve the spatial-temporal interpretation capabilities of the LiDAR detector by incorporating a dynamic prior, generated from a non-learnable motion estimation model. Specifically, Motion-Guided Feature Aggregation (MGFA) is proposed to utilize the object trajectory from previous and future motion states to model spatial-temporal correlations into gaussian heatmap over a driving sequence. This motion-based heatmap then guides the temporal feature fusion, enriching the proposed object features. Moreover, we design a Dual Correlation Weighting Module (DCWM) that effectively facilitates the interaction between past and prospective frames through scene- and channel-wise feature abstraction. In the end, a cascade cross-attention-based decoder is employed to refine the 3D prediction. We have conducted experiments on the Waymo and nuScenes datasets to demonstrate that the proposed framework achieves superior 3D detection performance with effective spatial-temporal feature learning.