InfraParis: A multi-modal and multi-task autonomous driving dataset

TL;DR

InfraParis is a comprehensive multi-modal dataset for autonomous driving, supporting RGB, depth, and infrared data across multiple tasks, aiming to improve model robustness in diverse and challenging scenarios.

Contribution

We introduce InfraParis, a new multi-modal, multi-task dataset for autonomous driving, enabling better training and evaluation of models in varied conditions.

Findings

Baseline models show varying performance across modalities.

InfraParis facilitates research on multi-modal sensor fusion.

The dataset covers diverse scenarios including nighttime and noise conditions.

Abstract

Current deep neural networks (DNNs) for autonomous driving computer vision are typically trained on specific datasets that only involve a single type of data and urban scenes. Consequently, these models struggle to handle new objects, noise, nighttime conditions, and diverse scenarios, which is essential for safety-critical applications. Despite ongoing efforts to enhance the resilience of computer vision DNNs, progress has been sluggish, partly due to the absence of benchmarks featuring multiple modalities. We introduce a novel and versatile dataset named InfraParis that supports multiple tasks across three modalities: RGB, depth, and infrared. We assess various state-of-the-art baseline techniques, encompassing models for the tasks of semantic segmentation, object detection, and depth estimation. More visualizations and the download link for InfraParis are available at \href{https://ensta-u2is.github.io/infraParis/}{https://ensta-u2is.github.io/infraParis/}.