Unsupervised Deraining: Where Asymmetric Contrastive Learning Meets Self-similarity

TL;DR

This paper introduces an unsupervised deraining method leveraging asymmetric contrastive learning that exploits intra-layer self-similarity and inter-layer exclusiveness, enabling effective rain removal without synthetic data.

Contribution

The proposed method uniquely combines non-local contrastive learning with an asymmetric loss to improve unsupervised rain and clean image separation, and introduces a large-scale real rainy image dataset.

Findings

Effective unsupervised deraining achieved without synthetic data

Enhanced discriminative decomposition via asymmetric contrastive loss

A new large-scale high-resolution rainy image dataset

Abstract

Most of the existing learning-based deraining methods are supervisedly trained on synthetic rainy-clean pairs. The domain gap between the synthetic and real rain makes them less generalized to complex real rainy scenes. Moreover, the existing methods mainly utilize the property of the image or rain layers independently, while few of them have considered their mutually exclusive relationship. To solve above dilemma, we explore the intrinsic intra-similarity within each layer and inter-exclusiveness between two layers and propose an unsupervised non-local contrastive learning (NLCL) deraining method. The non-local self-similarity image patches as the positives are tightly pulled together, rain patches as the negatives are remarkably pushed away, and vice versa. On one hand, the intrinsic self-similarity knowledge within positive/negative samples of each layer benefits us to discover more compact representation; on the other hand, the mutually exclusive property between the two layers enriches the discriminative decomposition. Thus, the internal self-similarity within each layer (similarity) and the external exclusive relationship of the two layers (dissimilarity) serving as a generic image prior jointly facilitate us to unsupervisedly differentiate the rain from clean image. We further discover that the intrinsic dimension of the non-local image patches is generally higher than that of the rain patches. This motivates us to design an asymmetric contrastive loss to precisely model the compactness discrepancy of the two layers for better discriminative decomposition. In addition, considering that the existing real rain datasets are of low quality, either small scale or downloaded from the internet, we collect a real large-scale dataset under various rainy kinds of weather that contains high-resolution rainy images.