# Robust Semantic Segmentation in Adverse Weather Conditions by means of Sensor Data Fusion

## TL;DR

This paper presents sensor data fusion combined with a Robust Learning Method to significantly improve semantic segmentation accuracy in adverse weather conditions for autonomous vehicles, without needing disturbance models.

## Contribution

It introduces a novel training approach, the RLM, and compares two sensor fusion architectures, demonstrating enhanced robustness in challenging weather scenarios.

## Key findings

- RLM greatly improves robustness in adverse weather
- Sensor fusion architectures outperform single sensors
- Good results achieved without disturbance models

## Abstract

A robust and reliable semantic segmentation in adverse weather conditions is very important for autonomous cars, but most state-of-the-art approaches only achieve high accuracy rates in optimal weather conditions. The reason is that they are only optimized for good weather conditions and given noise models. However, most of them fail, if data with unknown disturbances occur, and their performance decrease enormously. One possibility to still obtain reliable results is to observe the environment with different sensor types, such as camera and lidar, and to fuse the sensor data by means of neural networks, since different sensors behave differently in diverse weather conditions. Hence, the sensors can complement each other by means of an appropriate sensor data fusion. Nevertheless, the fusion-based approaches are still susceptible to disturbances and fail to classify disturbed image areas correctly. This problem can be solved by means of a special training method, the so called Robust Learning Method (RLM), a method by which the neural network learns to handle unknown noise. In this work, two different sensor fusion architectures for semantic segmentation are compared and evaluated on several datasets. Furthermore, it is shown that the RLM increases the robustness in adverse weather conditions enormously, and achieve good results although no disturbance model has been learned by the neural network.