Density-aware Single Image De-raining using a Multi-stream Dense Network

TL;DR

This paper introduces DID-MDN, a density-aware multi-stream neural network that automatically estimates rain density and effectively removes rain streaks from single images, improving de-raining performance.

Contribution

The paper proposes a novel density-aware multi-stream densely connected network for joint rain density estimation and de-raining, along with a new dataset with rain-density labels.

Findings

Significant improvements over state-of-the-art methods on synthetic and real datasets.

Effective automatic rain-density estimation guiding the de-raining process.

Enhanced removal of rain streaks with varying scales and shapes.

Abstract

Single image rain streak removal is an extremely challenging problem due to the presence of non-uniform rain densities in images. We present a novel density-aware multi-stream densely connected convolutional neural network-based algorithm, called DID-MDN, for joint rain density estimation and de-raining. The proposed method enables the network itself to automatically determine the rain-density information and then efficiently remove the corresponding rain-streaks guided by the estimated rain-density label. To better characterize rain-streaks with different scales and shapes, a multi-stream densely connected de-raining network is proposed which efficiently leverages features from different scales. Furthermore, a new dataset containing images with rain-density labels is created and used to train the proposed density-aware network. Extensive experiments on synthetic and real datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art methods. In addition, an ablation study is performed to demonstrate the improvements obtained by different modules in the proposed method. Code can be found at: https://github.com/hezhangsprinter