SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal

TL;DR

SemiDDM-Weather introduces a semi-supervised, all-in-one adverse weather removal framework utilizing a teacher-student network with a Wavelet Diffusion Model backbone, effectively handling limited labeled data and multiple weather conditions.

Contribution

It pioneers a semi-supervised, all-in-one adverse weather removal approach using a diffusion model with quality and content constraints for pseudo-label selection.

Findings

Outperforms fully supervised methods on synthetic and real datasets.

Maintains high visual quality in adverse weather removal.

Effectively handles multiple weather types with limited labeled data.

Abstract

Adverse weather removal aims to restore clear vision under adverse weather conditions. Existing methods are mostly tailored for specific weather types and rely heavily on extensive labeled data. In dealing with these two limitations, this paper presents a pioneering semi-supervised all-in-one adverse weather removal framework built on the teacher-student network with a Denoising Diffusion Model (DDM) as the backbone, termed SemiDDM-Weather. As for the design of DDM backbone in our SemiDDM-Weather, we adopt the SOTA Wavelet Diffusion Model-Wavediff with customized inputs and loss functions, devoted to facilitating the learning of many-to-one mapping distributions for efficient all-in-one adverse weather removal with limited label data. To mitigate the risk of misleading model training due to potentially inaccurate pseudo-labels generated by the teacher network in semi-supervised learning, we introduce quality assessment and content consistency constraints to screen the "optimal" outputs from the teacher network as the pseudo-labels, thus more effectively guiding the student network training with unlabeled data. Experimental results show that on both synthetic and real-world datasets, our SemiDDM-Weather consistently delivers high visual quality and superior adverse weather removal, even when compared to fully supervised competitors. Our code and pre-trained model are available at this repository.