Real-Time Environment Condition Classification for Autonomous Vehicles

TL;DR

This paper presents RECNet, a deep learning model that classifies outdoor weather and road conditions in real time, enhancing autonomous vehicle safety by enabling environment assessment without geo-fencing.

Contribution

It introduces a new taxonomy and label hierarchy for adverse-weather datasets, along with a semi-automated relabeling pipeline, and demonstrates improved classification performance in real-time.

Findings

RECNet outperforms baseline models by 16% in F1-Score.

The model operates at 20 Hz, suitable for real-time applications.

Enhanced dataset labeling improves environment condition classification.

Abstract

Current autonomous driving technologies are being rolled out in geo-fenced areas with well-defined operation conditions such as time of operation, area, weather conditions and road conditions. In this way, challenging conditions as adverse weather, slippery road or densely-populated city centers can be excluded. In order to lift the geo-fenced restriction and allow a more dynamic availability of autonomous driving functions, it is necessary for the vehicle to autonomously perform an environment condition assessment in real time to identify when the system cannot operate safely and either stop operation or require the resting passenger to take control. In particular, adverse-weather challenges are a fundamental limitation as sensor performance degenerates quickly, prohibiting the use of sensors such as cameras to locate and monitor road signs, pedestrians or other vehicles. To address this issue, we train a deep learning model to identify outdoor weather and dangerous road conditions, enabling a quick reaction to new situations and environments. We achieve this by introducing an improved taxonomy and label hierarchy for a state-of-the-art adverse-weather dataset, relabelling it with a novel semi-automated labeling pipeline. Using the novel proposed dataset and hierarchy, we train RECNet, a deep learning model for the classification of environment conditions from a single RGB frame. We outperform baseline models by relative 16% in F1- Score, while maintaining a real-time capable performance of 20 Hz.