Semantics-aware LiDAR-Only Pseudo Point Cloud Generation for 3D Object Detection

TL;DR

This paper introduces a LiDAR-only method that generates dense, semantically enriched pseudo point clouds to improve 3D object detection without relying on camera data.

Contribution

It proposes a novel framework combining scene semantics and a domain translator to create dense pseudo point clouds solely from LiDAR data, enhancing detection performance.

Findings

Up to 2.9% performance improvement on 3D detection tasks.

Comparable results to state-of-the-art LiDAR-only detectors on KITTI dataset.

Effective semantic-guided pseudo point cloud generation.

Abstract

Although LiDAR sensors are crucial for autonomous systems due to providing precise depth information, they struggle with capturing fine object details, especially at a distance, due to sparse and non-uniform data. Recent advances introduced pseudo-LiDAR, i.e., synthetic dense point clouds, using additional modalities such as cameras to enhance 3D object detection. We present a novel LiDAR-only framework that augments raw scans with denser pseudo point clouds by solely relying on LiDAR sensors and scene semantics, omitting the need for cameras. Our framework first utilizes a segmentation model to extract scene semantics from raw point clouds, and then employs a multi-modal domain translator to generate synthetic image segments and depth cues without real cameras. This yields a dense pseudo point cloud enriched with semantic information. We also introduce a new semantically guided projection method, which enhances detection performance by retaining only relevant pseudo points. We applied our framework to different advanced 3D object detection methods and reported up to 2.9% performance upgrade. We also obtained comparable results on the KITTI 3D object detection dataset, in contrast to other state-of-the-art LiDAR-only detectors.