Single Image Deraining using Scale-Aware Multi-Stage Recurrent Network

TL;DR

This paper introduces a scale-aware multi-stage recurrent neural network for single image deraining, effectively handling various rain streak sizes and reducing veiling effects, especially in heavy rain conditions.

Contribution

It proposes a novel parallel sub-network architecture that treats different rain streak scales separately, improving deraining performance over existing methods.

Findings

Outperforms state-of-the-art deraining methods on synthetic images.

Effective in removing heavy rain streaks and veiling effects.

Works well on real-world rainy images.

Abstract

Given a single input rainy image, our goal is to visually remove rain streaks and the veiling effect caused by scattering and transmission of rain streaks and rain droplets. We are particularly concerned with heavy rain, where rain streaks of various sizes and directions can overlap each other and the veiling effect reduces contrast severely. To achieve our goal, we introduce a scale-aware multi-stage convolutional neural network. Our main idea here is that different sizes of rain-streaks visually degrade the scene in different ways. Large nearby streaks obstruct larger regions and are likely to reflect specular highlights more prominently than smaller distant streaks. These different effects of different streaks have their own characteristics in their image features, and thus need to be treated differently. To realize this, we create parallel sub-networks that are trained and made aware of these different scales of rain streaks. To our knowledge, this idea of parallel sub-networks that treats the same class of objects according to their unique sub-classes is novel, particularly in the context of rain removal. To verify our idea, we conducted experiments on both synthetic and real images, and found that our method is effective and outperforms the state-of-the-art methods.