AWRaCLe: All-Weather Image Restoration using Visual In-Context Learning

TL;DR

AWRaCLe introduces a novel all-weather image restoration method that leverages visual in-context learning with degradation-specific context extraction and fusion, significantly improving performance under adverse weather conditions.

Contribution

The paper presents a new approach that utilizes degradation-specific visual context to guide image restoration, incorporating CLIP-based features and attention mechanisms for enhanced all-weather performance.

Findings

Outperforms existing methods in all-weather image restoration tasks.

Effectively utilizes degradation context to improve restoration quality.

Achieves state-of-the-art results on benchmark datasets.

Abstract

All-Weather Image Restoration (AWIR) under adverse weather conditions is a challenging task due to the presence of different types of degradations. Prior research in this domain relies on extensive training data but lacks the utilization of additional contextual information for restoration guidance. Consequently, the performance of existing methods is limited by the degradation cues that are learnt from individual training samples. Recent advancements in visual in-context learning have introduced generalist models that are capable of addressing multiple computer vision tasks simultaneously by using the information present in the provided context as a prior. In this paper, we propose All-Weather Image Restoration using Visual In-Context Learning (AWRaCLe), a novel approach for AWIR that innovatively utilizes degradation-specific visual context information to steer the image restoration process. To achieve this, AWRaCLe incorporates Degradation Context Extraction (DCE) and Context Fusion (CF) to seamlessly integrate degradation-specific features from the context into an image restoration network. The proposed DCE and CF blocks leverage CLIP features and incorporate attention mechanisms to adeptly learn and fuse contextual information. These blocks are specifically designed for visual in-context learning under all-weather conditions and are crucial for effective context utilization. Through extensive experiments, we demonstrate the effectiveness of AWRaCLe for all-weather restoration and show that our method advances the state-of-the-art in AWIR.