Robust 3D Object Detection using Probabilistic Point Clouds from Single-Photon LiDARs

TL;DR

This paper introduces Probabilistic Point Clouds (PPC), a new 3D scene representation that encodes measurement uncertainty in LiDAR data, leading to more robust 3D object detection especially in challenging scenarios.

Contribution

The paper proposes PPC, a novel point cloud representation with uncertainty attributes, and inference methods that improve robustness in 3D detection over traditional approaches.

Findings

PPC-based methods outperform baselines in challenging scenarios.

PPC improves detection accuracy for small, distant, and low-albedo objects.

PPC-based inference is computationally lightweight and versatile.

Abstract

LiDAR-based 3D sensors provide point clouds, a canonical 3D representation used in various scene understanding tasks. Modern LiDARs face key challenges in several real-world scenarios, such as long-distance or low-albedo objects, producing sparse or erroneous point clouds. These errors, which are rooted in the noisy raw LiDAR measurements, get propagated to downstream perception models, resulting in potentially severe loss of accuracy. This is because conventional 3D processing pipelines do not retain any uncertainty information from the raw measurements when constructing point clouds. We propose Probabilistic Point Clouds (PPC), a novel 3D scene representation where each point is augmented with a probability attribute that encapsulates the measurement uncertainty (or confidence) in the raw data. We further introduce inference approaches that leverage PPC for robust 3D object detection; these methods are versatile and can be used as computationally lightweight drop-in modules in 3D inference pipelines. We demonstrate, via both simulations and real captures, that PPC-based 3D inference methods outperform several baselines using LiDAR as well as camera-LiDAR fusion models, across challenging indoor and outdoor scenarios involving small, distant, and low-albedo objects, as well as strong ambient light. Our project webpage is at https://bhavyagoyal.github.io/ppc .