Rethinking Image Deraining via Rain Streaks and Vapors

TL;DR

This paper introduces a novel rain image formation model that treats rain streaks and vapors as transmission mediums, and proposes a CNN-based framework to improve single image deraining by accurately modeling these components.

Contribution

It reformulates rain streaks as transmission mediums with vapors, and develops a joint CNN model to better predict rain components for image restoration.

Findings

Outperforms state-of-the-art deraining methods on benchmark datasets

Effectively models rain streaks and vapors as transmission mediums

Uses multi-scale spatial pyramid pooling for vapor transmission prediction

Abstract

Single image deraining regards an input image as a fusion of a background image, a transmission map, rain streaks, and atmosphere light. While advanced models are proposed for image restoration (i.e., background image generation), they regard rain streaks with the same properties as background rather than transmission medium. As vapors (i.e., rain streaks accumulation or fog-like rain) are conveyed in the transmission map to model the veiling effect, the fusion of rain streaks and vapors do not naturally reflect the rain image formation. In this work, we reformulate rain streaks as transmission medium together with vapors to model rain imaging. We propose an encoder-decoder CNN named as SNet to learn the transmission map of rain streaks. As rain streaks appear with various shapes and directions, we use ShuffleNet units within SNet to capture their anisotropic representations. As vapors are brought by rain streaks, we propose a VNet containing spatial pyramid pooling (SSP) to predict the transmission map of vapors in multi-scales based on that of rain streaks. Meanwhile, we use an encoder CNN named ANet to estimate atmosphere light. The SNet, VNet, and ANet are jointly trained to predict transmission maps and atmosphere light for rain image restoration. Extensive experiments on the benchmark datasets demonstrate the effectiveness of the proposed visual model to predict rain streaks and vapors. The proposed deraining method performs favorably against state-of-the-art deraining approaches.