Real3D-Aug: Point Cloud Augmentation by Placing Real Objects with Occlusion Handling for 3D Detection and Segmentation

TL;DR

Real3D-Aug is a data augmentation technique for 3D lidar point clouds that reuses real data, intelligently places objects with occlusion handling, and improves detection and segmentation performance, especially for rare classes.

Contribution

It introduces a novel augmentation framework that leverages real data with automatic placement and occlusion handling for enhanced 3D detection and segmentation.

Findings

Achieves 6.65% AP gain for hard pedestrian detection in KITTI.

Attains 2.14 mean IoU improvement in SemanticKITTI segmentation.

Proves competitive and effective in training top models for 3D tasks.

Abstract

Object detection and semantic segmentation with the 3D lidar point cloud data require expensive annotation. We propose a data augmentation method that takes advantage of already annotated data multiple times. We propose an augmentation framework that reuses real data, automatically finds suitable placements in the scene to be augmented, and handles occlusions explicitly. Due to the usage of the real data, the scan points of newly inserted objects in augmentation sustain the physical characteristics of the lidar, such as intensity and raydrop. The pipeline proves competitive in training top-performing models for 3D object detection and semantic segmentation. The new augmentation provides a significant performance gain in rare and essential classes, notably 6.65% average precision gain for "Hard" pedestrian class in KITTI object detection or 2.14 mean IoU gain in the SemanticKITTI segmentation challenge over the state of the art.