Enhanced Semantic Segmentation Pipeline for WeatherProof Dataset Challenge

TL;DR

This paper presents a winning semantic segmentation pipeline for the WeatherProof Dataset Challenge, combining improved models, new dataset, data augmentation, and ensemble strategies to achieve top performance.

Contribution

It introduces an enhanced segmentation pipeline with pretrained backbones, language guidance, a new dataset, advanced augmentation, and ensemble methods, setting a new benchmark.

Findings

Achieved 1st place in the challenge leaderboard.

Improved segmentation accuracy with pretrained Depth Anything and language guidance.

Enhanced robustness through data augmentation and ensemble techniques.

Abstract

This report describes the winning solution to the WeatherProof Dataset Challenge (CVPR 2024 UG2+ Track 3). Details regarding the challenge are available at https://cvpr2024ug2challenge.github.io/track3.html. We propose an enhanced semantic segmentation pipeline for this challenge. Firstly, we improve semantic segmentation models, using backbone pretrained with Depth Anything to improve UperNet model and SETRMLA model, and adding language guidance based on both weather and category information to InternImage model. Secondly, we introduce a new dataset WeatherProofExtra with wider viewing angle and employ data augmentation methods, including adverse weather and super-resolution. Finally, effective training strategies and ensemble method are applied to improve final performance further. Our solution is ranked 1st on the final leaderboard. Code will be available at https://github.com/KaneiGi/WeatherProofChallenge.