Synth It Like KITTI: Synthetic Data Generation for Object Detection in Driving Scenarios

TL;DR

This paper presents a synthetic data generation pipeline using CARLA for 3D LiDAR object detection in autonomous driving, achieving strong transferability to real-world KITTI data through domain randomization and sensor analysis.

Contribution

It introduces a novel synthetic dataset generation method that improves transferability of LiDAR-based object detection models to real-world data.

Findings

Synthetic data enables effective training of 3D object detectors.

Domain randomization reduces the domain gap between synthetic and real data.

Fine-tuning on small real datasets nearly matches or surpasses real data training baselines.

Abstract

An important factor in advancing autonomous driving systems is simulation. Yet, there is rather small progress for transferability between the virtual and real world. We revisit this problem for 3D object detection on LiDAR point clouds and propose a dataset generation pipeline based on the CARLA simulator. Utilizing domain randomization strategies and careful modeling, we are able to train an object detector on the synthetic data and demonstrate strong generalization capabilities to the KITTI dataset. Furthermore, we compare different virtual sensor variants to gather insights, which sensor attributes can be responsible for the prevalent domain gap. Finally, fine-tuning with a small portion of real data almost matches the baseline and with the full training set slightly surpasses it.