Towards Streaming LiDAR Object Detection with Point Clouds as Egocentric Sequences

TL;DR

This paper introduces PFCF, a hybrid LiDAR object detection method that combines fast polar processing with accurate Cartesian reasoning, achieving high accuracy and low latency for autonomous driving.

Contribution

The paper proposes PFCF, a novel hybrid detector that integrates polar and Cartesian processing with a custom streaming backbone, improving speed and accuracy in LiDAR-based detection.

Findings

Surpasses prior streaming methods by 10% mAP on Waymo dataset

Matches full-scan accuracy at twice the update rate

Uses a distortion-robust, parameter-efficient backbone

Abstract

Accurate and low-latency 3D object detection is essential for autonomous driving, where safety hinges on both rapid response and reliable perception. While rotating LiDAR sensors are widely adopted for their robustness and fidelity, current detectors face a trade-off: streaming methods process partial polar sectors on the fly for fast updates but suffer from limited visibility, cross-sector dependencies, and distortions from retrofitted Cartesian designs, whereas full-scan methods achieve higher accuracy but are bottlenecked by the inherent latency of a LiDAR revolution. We propose Polar-Fast-Cartesian-Full (PFCF), a hybrid detector that combines fast polar processing for intra-sector feature extraction with accurate Cartesian reasoning for full-scene understanding. Central to PFCF is a custom Mamba SSM-based streaming backbone with dimensionally-decomposed convolutions that avoids distortion-heavy planes, enabling parameter-efficient, translation-invariant, and distortion-robust polar representation learning. Local sector features are extracted via this backbone, then accumulated into a sector feature buffer to enable efficient inter-sector communication through a full-scan backbone. PFCF establishes a new Pareto frontier on the Waymo Open dataset, surpassing prior streaming baselines by 10% mAP and matching full-scan accuracy at twice the update rate. Code is available at \href{https://github.com/meilongzhang/Polar-Hierarchical-Mamba}{https://github.com/meilongzhang/Polar-Hierarchical-Mamba}.