# REHEARSE-3D: A Multi-Modal Emulated Rain Dataset for 3D Point Cloud De-Raining

## TL;DR

REHEARSE-3D is a large-scale dataset for improving 3D point cloud de-raining in autonomous driving systems during heavy rain.

## Contribution

The paper introduces REHEARSE-3D, the largest point-wise annotated multi-modal emulated rain dataset with high-resolution LiDAR and 4D RADAR data.

## Key findings

- REHEARSE-3D contains 9.2 billion annotated points, making it the largest of its kind.
- The dataset includes LiDAR-256 and 4D RADAR data collected in both day and night under controlled weather.
- SalsaNext and 3D-OutDet models achieved over 94% IoU for raindrop detection in benchmark tests.

## Abstract

Sensor degradation poses a significant challenge in autonomous driving. During heavy rainfall, interference from raindrops can adversely affect the quality of LiDAR point clouds, resulting in, for instance, inaccurate point measurements. This, in turn, can potentially lead to safety concerns if autonomous driving systems are not weather-aware, i.e., if they are unable to discern such changes. In this study, we release a new, large-scale, multi-modal emulated rain dataset, REHEARSE-3D, to promote research advancements in 3D point cloud de-raining. Distinct from the most relevant competitors, our dataset is unique in several respects. First, it is the largest point-wise annotated dataset (9.2 billion annotated points), and second, it is the only one with high-resolution LiDAR data (LiDAR-256) enriched with 4D RADAR point clouds logged in both daytime and nighttime conditions in a controlled weather environment. Furthermore, REHEARSE-3D involves rain-characteristic information, which is of significant value not only for sensor noise modeling but also for analyzing the impact of weather at the point level. Leveraging REHEARSE-3D, we benchmark raindrop detection and removal in fused LiDAR and 4D RADAR point clouds. Our comprehensive study further evaluates the performance of various statistical and deep learning models, where SalsaNext and 3D-OutDet achieve above 94% IoU for raindrop detection.