Rethinking Data Augmentation for Robust LiDAR Semantic Segmentation in Adverse Weather

TL;DR

This paper analyzes the effects of adverse weather on LiDAR semantic segmentation and proposes novel data augmentation techniques that significantly improve robustness and achieve state-of-the-art results.

Contribution

It introduces targeted data augmentation methods, Selective Jittering and Learnable Point Drop, to address geometric perturbation and point drop caused by adverse weather.

Findings

Achieved 39.5 mIoU on SemanticKITTI-to-SemanticSTF benchmark

Improved baseline performance by 8.1 percentage points

Established new state-of-the-art results

Abstract

Existing LiDAR semantic segmentation methods often struggle with performance declines in adverse weather conditions. Previous work has addressed this issue by simulating adverse weather or employing universal data augmentation during training. However, these methods lack a detailed analysis and understanding of how adverse weather negatively affects LiDAR semantic segmentation performance. Motivated by this issue, we identified key factors of adverse weather and conducted a toy experiment to pinpoint the main causes of performance degradation: (1) Geometric perturbation due to refraction caused by fog or droplets in the air and (2) Point drop due to energy absorption and occlusions. Based on these findings, we propose new strategic data augmentation techniques. First, we introduced a Selective Jittering (SJ) that jitters points in the random range of depth (or angle) to mimic geometric perturbation. Additionally, we developed a Learnable Point Drop (LPD) to learn vulnerable erase patterns with a Deep Q-Learning Network to approximate the point drop phenomenon from adverse weather conditions. Without precise weather simulation, these techniques strengthen the LiDAR semantic segmentation model by exposing it to vulnerable conditions identified by our data-centric analysis. Experimental results confirmed the suitability of the proposed data augmentation methods for enhancing robustness against adverse weather conditions. Our method achieves a notable 39.5 mIoU on the SemanticKITTI-to-SemanticSTF benchmark, improving the baseline by 8.1\%p and establishing a new state-of-the-art. Our code will be released at \url{https://github.com/engineerJPark/LiDARWeather}.