3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection

TL;DR

This paper introduces 3D-VField, a novel adversarial data augmentation technique that deforms point clouds to improve the robustness and generalization of 3D object detectors across different domains and sensor types.

Contribution

The paper proposes 3D-VField, a new vector field-based augmentation method that plausibly deforms point clouds without adding or removing points, enhancing out-of-domain detection performance.

Findings

Significant improvement in generalization to out-of-domain data.

Effective augmentation across LiDAR and ToF sensor data.

Introduction of CrashD, a synthetic dataset for damaged and rare cars.

Abstract

As 3D object detection on point clouds relies on the geometrical relationships between the points, non-standard object shapes can hinder a method's detection capability. However, in safety-critical settings, robustness to out-of-domain and long-tail samples is fundamental to circumvent dangerous issues, such as the misdetection of damaged or rare cars. In this work, we substantially improve the generalization of 3D object detectors to out-of-domain data by deforming point clouds during training. We achieve this with 3D-VField: a novel data augmentation method that plausibly deforms objects via vector fields learned in an adversarial fashion. Our approach constrains 3D points to slide along their sensor view rays while neither adding nor removing any of them. The obtained vectors are transferable, sample-independent and preserve shape and occlusions. Despite training only on a standard dataset, such as KITTI, augmenting with our vector fields significantly improves the generalization to differently shaped objects and scenes. Towards this end, we propose and share CrashD: a synthetic dataset of realistic damaged and rare cars, with a variety of crash scenarios. Extensive experiments on KITTI, Waymo, our CrashD and SUN RGB-D show the generalizability of our techniques to out-of-domain data, different models and sensors, namely LiDAR and ToF cameras, for both indoor and outdoor scenes. Our CrashD dataset is available at https://crashd-cars.github.io.