Boosting Single-Frame 3D Object Detection by Simulating Multi-Frame Point Clouds

TL;DR

This paper introduces SMF-SSD, a training framework that enables single-frame 3D object detectors to mimic multi-frame detectors' features and responses, significantly improving detection accuracy without multi-frame input during inference.

Contribution

The paper proposes a novel training approach and architecture, including multi-view dense object fusion and knowledge distillation techniques, to enhance single-frame 3D detection performance.

Findings

SMF-SSD outperforms state-of-the-art single-frame detectors on Waymo dataset.

The approach improves detection accuracy across all object classes and difficulty levels.

Knowledge transfer techniques effectively simulate multi-frame features in single-frame detectors.

Abstract

To boost a detector for single-frame 3D object detection, we present a new approach to train it to simulate features and responses following a detector trained on multi-frame point clouds. Our approach needs multi-frame point clouds only when training the single-frame detector, and once trained, it can detect objects with only single-frame point clouds as inputs during the inference. We design a novel Simulated Multi-Frame Single-Stage object Detector (SMF-SSD) framework to realize the approach: multi-view dense object fusion to densify ground-truth objects to generate a multi-frame point cloud; self-attention voxel distillation to facilitate one-to-many knowledge transfer from multi- to single-frame voxels; multi-scale BEV feature distillation to transfer knowledge in low-level spatial and high-level semantic BEV features; and adaptive response distillation to activate single-frame responses of high confidence and accurate localization. Experimental results on the Waymo test set show that our SMF-SSD consistently outperforms all state-of-the-art single-frame 3D object detectors for all object classes of difficulty levels 1 and 2 in terms of both mAP and mAPH.