MSF: Motion-guided Sequential Fusion for Efficient 3D Object Detection from Point Cloud Sequences

TL;DR

This paper introduces MSF, a motion-guided sequential fusion method for efficient and accurate 3D object detection from point cloud sequences, leveraging object motion continuity to reduce redundant computation.

Contribution

The paper proposes a novel motion-guided fusion framework with a bidirectional feature aggregation module, improving efficiency and accuracy over existing multi-frame detectors.

Findings

Achieves 83.12% mAP on Waymo LEVEL1 test set

Runs in several milliseconds with millions of points processed

Outperforms existing multi-frame detectors in efficiency and accuracy

Abstract

Point cloud sequences are commonly used to accurately detect 3D objects in applications such as autonomous driving. Current top-performing multi-frame detectors mostly follow a Detect-and-Fuse framework, which extracts features from each frame of the sequence and fuses them to detect the objects in the current frame. However, this inevitably leads to redundant computation since adjacent frames are highly correlated. In this paper, we propose an efficient Motion-guided Sequential Fusion (MSF) method, which exploits the continuity of object motion to mine useful sequential contexts for object detection in the current frame. We first generate 3D proposals on the current frame and propagate them to preceding frames based on the estimated velocities. The points-of-interest are then pooled from the sequence and encoded as proposal features. A novel Bidirectional Feature Aggregation (BiFA) module is further proposed to facilitate the interactions of proposal features across frames. Besides, we optimize the point cloud pooling by a voxel-based sampling technique so that millions of points can be processed in several milliseconds. The proposed MSF method achieves not only better efficiency than other multi-frame detectors but also leading accuracy, with 83.12% and 78.30% mAP on the LEVEL1 and LEVEL2 test sets of Waymo Open Dataset, respectively. Codes can be found at \url{https://github.com/skyhehe123/MSF}.