Uncertainty-Driven Multi-Scale Feature Fusion Network for Real-time Image Deraining

TL;DR

This paper introduces UMFFNet, a real-time image deraining network that incorporates uncertainty estimation and multi-scale feature fusion to improve rain removal performance on resource-limited devices.

Contribution

The paper proposes a novel uncertainty-driven multi-scale feature fusion network with an uncertainty feature fusion block for enhanced rain removal and uncertainty estimation.

Findings

Achieves superior deraining performance compared to state-of-the-art methods.

Operates efficiently with fewer parameters suitable for resource-constrained devices.

Effectively reduces prediction errors by leveraging uncertainty information.

Abstract

Visual-based measurement systems are frequently affected by rainy weather due to the degradation caused by rain streaks in captured images, and existing imaging devices struggle to address this issue in real-time. While most efforts leverage deep networks for image deraining and have made progress, their large parameter sizes hinder deployment on resource-constrained devices. Additionally, these data-driven models often produce deterministic results, without considering their inherent epistemic uncertainty, which can lead to undesired reconstruction errors. Well-calibrated uncertainty can help alleviate prediction errors and assist measurement devices in mitigating risks and improving usability. Therefore, we propose an Uncertainty-Driven Multi-Scale Feature Fusion Network (UMFFNet) that learns the probability mapping distribution between paired images to estimate uncertainty. Specifically, we introduce an uncertainty feature fusion block (UFFB) that utilizes uncertainty information to dynamically enhance acquired features and focus on blurry regions obscured by rain streaks, reducing prediction errors. In addition, to further boost the performance of UMFFNet, we fused feature information from multiple scales to guide the network for efficient collaborative rain removal. Extensive experiments demonstrate that UMFFNet achieves significant performance improvements with few parameters, surpassing state-of-the-art image deraining methods.