DTDN: Dual-task De-raining Network

TL;DR

The paper introduces DTDN, a dual-task network combining GAN and CNN to effectively remove rain streaks while preserving image details, outperforming recent methods on benchmarks and real images.

Contribution

It presents a novel end-to-end dual-task network with a unique training algorithm for rain removal and detail preservation, addressing the balance between these objectives.

Findings

Outperforms state-of-the-art de-raining methods on benchmarks.

Effectively removes rain streaks while maintaining image details.

Enriches datasets to better represent real rain streaks.

Abstract

Removing rain streaks from rainy images is necessary for many tasks in computer vision, such as object detection and recognition. It needs to address two mutually exclusive objectives: removing rain streaks and reserving realistic details. Balancing them is critical for de-raining methods. We propose an end-to-end network, called dual-task de-raining network (DTDN), consisting of two sub-networks: generative adversarial network (GAN) and convolutional neural network (CNN), to remove rain streaks via coordinating the two mutually exclusive objectives self-adaptively. DTDN-GAN is mainly used to remove structural rain streaks, and DTDN-CNN is designed to recover details in original images. We also design a training algorithm to train these two sub-networks of DTDN alternatively, which share same weights but use different training sets. We further enrich two existing datasets to approximate the distribution of real rain streaks. Experimental results show that our method outperforms several recent state-of-the-art methods, based on both benchmark testing datasets and real rainy images.