Joint Depth Estimation and Mixture of Rain Removal From a Single Image

TL;DR

This paper introduces DEMore-Net, a joint depth estimation and mixture of rain removal network that effectively removes complex rain artifacts from a single image by leveraging depth information and a novel normalization technique.

Contribution

The study presents a novel joint learning framework that combines depth estimation with rain removal and introduces Hybrid Normalization Block to improve deraining performance.

Findings

DEMores-Net outperforms existing methods on synthetic datasets.

The joint approach effectively handles various types of rain artifacts.

Hybrid Normalization Block enhances deraining results.

Abstract

Rainy weather significantly deteriorates the visibility of scene objects, particularly when images are captured through outdoor camera lenses or windshields. Through careful observation of numerous rainy photos, we have found that the images are generally affected by various rainwater artifacts such as raindrops, rain streaks, and rainy haze, which impact the image quality from both near and far distances, resulting in a complex and intertwined process of image degradation. However, current deraining techniques are limited in their ability to address only one or two types of rainwater, which poses a challenge in removing the mixture of rain (MOR). In this study, we propose an effective image deraining paradigm for Mixture of rain REmoval, called DEMore-Net, which takes full account of the MOR effect. Going beyond the existing deraining wisdom, DEMore-Net is a joint learning paradigm that integrates depth estimation and MOR removal tasks to achieve superior rain removal. The depth information can offer additional meaningful guidance information based on distance, thus better helping DEMore-Net remove different types of rainwater. Moreover, this study explores normalization approaches in image deraining tasks and introduces a new Hybrid Normalization Block (HNB) to enhance the deraining performance of DEMore-Net. Extensive experiments conducted on synthetic datasets and real-world MOR photos fully validate the superiority of the proposed DEMore-Net. Code is available at https://github.com/yz-wang/DEMore-Net.