Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains

TL;DR

This paper introduces CoIC, a novel deraining algorithm that leverages joint rain- and detail-aware representations and a context-based modulation mechanism to improve model performance across diverse rainy images.

Contribution

It proposes a joint rain-/detail-aware contrastive learning strategy combined with a context-based modulation mechanism, advancing model robustness on mixed rainy datasets.

Findings

CoIC significantly improves deraining performance on CNN and Transformer models.

The method effectively models relationships between datasets and assesses rain and detail impacts.

Enhanced deraining results when real-world datasets are incorporated.

Abstract

Recent advances in image deraining have focused on training powerful models on mixed multiple datasets comprising diverse rain types and backgrounds. However, this approach tends to overlook the inherent differences among rainy images, leading to suboptimal results. To overcome this limitation, we focus on addressing various rainy images by delving into meaningful representations that encapsulate both the rain and background components. Leveraging these representations as instructive guidance, we put forth a Context-based Instance-level Modulation (CoI-M) mechanism adept at efficiently modulating CNN- or Transformer-based models. Furthermore, we devise a rain-/detail-aware contrastive learning strategy to help extract joint rain-/detail-aware representations. By integrating CoI-M with the rain-/detail-aware Contrastive learning, we develop CoIC, an innovative and potent algorithm tailored for training models on mixed datasets. Moreover, CoIC offers insight into modeling relationships of datasets, quantitatively assessing the impact of rain and details on restoration, and unveiling distinct behaviors of models given diverse inputs. Extensive experiments validate the efficacy of CoIC in boosting the deraining ability of CNN and Transformer models. CoIC also enhances the deraining prowess remarkably when real-world dataset is included.