Enhancing autonomous vehicle safety in rain: a data-centric approach for clear vision

TL;DR

This paper presents a deep learning-based vision model trained on a simulated dataset to remove rain effects from camera images, significantly improving autonomous vehicle steering accuracy in rainy conditions.

Contribution

It introduces a novel data-centric approach using simulation data and a specialized training scheme to enhance rain removal in vehicle vision systems.

Findings

Improved steering accuracy in rainy conditions

Effective rain pattern removal from camera images

Demonstrated potential for safer autonomous navigation

Abstract

Autonomous vehicles face significant challenges in navigating adverse weather, particularly rain, due to the visual impairment of camera-based systems. In this study, we leveraged contemporary deep learning techniques to mitigate these challenges, aiming to develop a vision model that processes live vehicle camera feeds to eliminate rain-induced visual hindrances, yielding visuals closely resembling clear, rain-free scenes. Using the Car Learning to Act (CARLA) simulation environment, we generated a comprehensive dataset of clear and rainy images for model training and testing. In our model, we employed a classic encoder-decoder architecture with skip connections and concatenation operations. It was trained using novel batching schemes designed to effectively distinguish high-frequency rain patterns from low-frequency scene features across successive image frames. To evaluate the model performance, we integrated it with a steering module that processes front-view images as input. The results demonstrated notable improvements in steering accuracy, underscoring the model's potential to enhance navigation safety and reliability in rainy weather conditions.