Teaching Tailored to Talent: Adverse Weather Restoration via Prompt Pool and Depth-Anything Constraint

TL;DR

This paper introduces T3-DiffWeather, a novel weather restoration method using prompt pools and scene constraints, achieving state-of-the-art results in handling complex weather degradations efficiently.

Contribution

The paper proposes a prompt pool-based approach with scene-specific constraints and contrastive loss, enhancing adaptability and performance in adverse weather restoration tasks.

Findings

Achieves state-of-the-art performance on synthetic and real-world datasets.

Outperforms existing diffusion techniques in computational efficiency.

Effectively handles complex and unforeseen weather degradations.

Abstract

Recent advancements in adverse weather restoration have shown potential, yet the unpredictable and varied combinations of weather degradations in the real world pose significant challenges. Previous methods typically struggle with dynamically handling intricate degradation combinations and carrying on background reconstruction precisely, leading to performance and generalization limitations. Drawing inspiration from prompt learning and the "Teaching Tailored to Talent" concept, we introduce a novel pipeline, T3-DiffWeather. Specifically, we employ a prompt pool that allows the network to autonomously combine sub-prompts to construct weather-prompts, harnessing the necessary attributes to adaptively tackle unforeseen weather input. Moreover, from a scene modeling perspective, we incorporate general prompts constrained by Depth-Anything feature to provide the scene-specific condition for the diffusion process. Furthermore, by incorporating contrastive prompt loss, we ensures distinctive representations for both types of prompts by a mutual pushing strategy. Experimental results demonstrate that our method achieves state-of-the-art performance across various synthetic and real-world datasets, markedly outperforming existing diffusion techniques in terms of computational efficiency.