Multi-Task Learning Enhanced Single Image De-Raining

TL;DR

This paper introduces MENET, a multi-task convolutional neural network based on U-Net, that leverages semantic constraints and dynamic weighting to effectively remove rain streaks from single images, outperforming state-of-the-art methods.

Contribution

It proposes a novel multi-task regression model with dynamic weighting algorithms for improved rain removal in images, integrating content, edge, and texture constraints.

Findings

Superior performance on synthetic and real rain images

Effective removal of heavy rain and streak accumulation

Outperforms several state-of-the-art approaches

Abstract

Rain removal in images is an important task in computer vision filed and attracting attentions of more and more people. In this paper, we address a non-trivial issue of removing visual effect of rain streak from a single image. Differing from existing work, our method combines various semantic constraint task in a proposed multi-task regression model for rain removal. These tasks reinforce the model's capabilities from the content, edge-aware, and local texture similarity respectively. To further improve the performance of multi-task learning, we also present two simple but powerful dynamic weighting algorithms. The proposed multi-task enhanced network (MENET) is a powerful convolutional neural network based on U-Net for rain removal research, with a specific focus on utilize multiple tasks constraints and exploit the synergy among them to facilitate the model's rain removal capacity. It is noteworthy that the adaptive weighting scheme has further resulted in improved network capability. We conduct several experiments on synthetic and real rain images, and achieve superior rain removal performance over several selected state-of-the-art (SOTA) approaches. The overall effect of our method is impressive, even in the decomposition of heavy rain and rain streak accumulation.The source code and some results can be found at:https://github.com/SumiHui/MENET.