Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping

TL;DR

Paris-CARLA-3D offers a combined real and synthetic outdoor point cloud dataset to advance 3D mapping tasks like segmentation and scene completion, facilitating transfer learning between simulated and real environments.

Contribution

This dataset uniquely pairs real and synthetic outdoor point clouds with aligned sensor platforms and annotations, enabling research on transfer learning and challenging 3D mapping tasks.

Findings

Baseline performance established for semantic segmentation.

Effective transfer methods demonstrated from synthetic to real data.

Dataset challenges highlight areas for future improvement.

Abstract

Paris-CARLA-3D is a dataset of several dense colored point clouds of outdoor environments built by a mobile LiDAR and camera system. The data are composed of two sets with synthetic data from the open source CARLA simulator (700 million points) and real data acquired in the city of Paris (60 million points), hence the name Paris-CARLA-3D. One of the advantages of this dataset is to have simulated the same LiDAR and camera platform in the open source CARLA simulator as the one used to produce the real data. In addition, manual annotation of the classes using the semantic tags of CARLA was performed on the real data, allowing the testing of transfer methods from the synthetic to the real data. The objective of this dataset is to provide a challenging dataset to evaluate and improve methods on difficult vision tasks for the 3D mapping of outdoor environments: semantic segmentation, instance segmentation, and scene completion. For each task, we describe the evaluation protocol as well as the experiments carried out to establish a baseline.