A Two-Stage Adverse Weather Semantic Segmentation Method for WeatherProof Challenge CVPR 2024 Workshop UG2+

TL;DR

This paper introduces a two-stage deep learning approach for semantic segmentation in adverse weather, utilizing video deraining and pseudo ground truths to improve model robustness and achieve competitive results.

Contribution

The novel two-stage framework combines video deraining and pseudo ground truth generation with a state-of-the-art segmentation network for adverse weather conditions.

Findings

Achieved a 0.43 mIoU score in the WeatherProof Challenge

Secured 4th place in the CVPR 2024 UG2+ workshop

Demonstrated robustness to degraded weather data

Abstract

This technical report presents our team's solution for the WeatherProof Dataset Challenge: Semantic Segmentation in Adverse Weather at CVPR'24 UG2+. We propose a two-stage deep learning framework for this task. In the first stage, we preprocess the provided dataset by concatenating images into video sequences. Subsequently, we leverage a low-rank video deraining method to generate high-fidelity pseudo ground truths. These pseudo ground truths offer superior alignment compared to the original ground truths, facilitating model convergence during training. In the second stage, we employ the InternImage network to train for the semantic segmentation task using the generated pseudo ground truths. Notably, our meticulously designed framework demonstrates robustness to degraded data captured under adverse weather conditions. In the challenge, our solution achieved a competitive score of 0.43 on the Mean Intersection over Union (mIoU) metric, securing a respectable rank of 4th.