Dense Road Surface Grip Map Prediction from Multimodal Image Data

TL;DR

This paper presents a convolutional neural network that predicts dense road grip maps from multimodal sensor data, enhancing autonomous vehicle safety in slippery conditions by accurately modeling road surface grip.

Contribution

It introduces a multimodal sensor fusion approach for dense grip map prediction, improving accuracy over single modality models in autonomous driving scenarios.

Findings

Fused sensor data improves grip prediction accuracy.

RGB and LiDAR alone provide strong baselines.

Multimodal fusion enhances grip map detail and reliability.

Abstract

Slippery road weather conditions are prevalent in many regions and cause a regular risk for traffic. Still, there has been less research on how autonomous vehicles could detect slippery driving conditions on the road to drive safely. In this work, we propose a method to predict a dense grip map from the area in front of the car, based on postprocessed multimodal sensor data. We trained a convolutional neural network to predict pixelwise grip values from fused RGB camera, thermal camera, and LiDAR reflectance images, based on weakly supervised ground truth from an optical road weather sensor. The experiments show that it is possible to predict dense grip values with good accuracy from the used data modalities as the produced grip map follows both ground truth measurements and local weather conditions, such as snowy areas on the road. The model using only the RGB camera or LiDAR reflectance modality provided good baseline results for grip prediction accuracy while using models fusing the RGB camera, thermal camera, and LiDAR modalities improved the grip predictions significantly.