Semantics and Content Matter: Towards Multi-Prior Hierarchical Mamba for Image Deraining

TL;DR

This paper introduces the Multi-Prior Hierarchical Mamba (MPHM) network, a novel image deraining architecture that combines semantic and structural priors with a hierarchical feature module, achieving state-of-the-art results and better generalization.

Contribution

The paper proposes a new hierarchical network that integrates macro-semantic and micro-structural priors with a fusion strategy for improved image deraining performance.

Findings

Achieves 0.57 dB PSNR gain on Rain200H dataset.

Demonstrates superior generalization on real-world rainy scenes.

Outperforms existing methods in state-of-the-art benchmarks.

Abstract

Rain significantly degrades the performance of computer vision systems, particularly in applications like autonomous driving and video surveillance. While existing deraining methods have made considerable progress, they often struggle with fidelity of semantic and spatial details. To address these limitations, we propose the Multi-Prior Hierarchical Mamba (MPHM) network for image deraining. This novel architecture synergistically integrates macro-semantic textual priors (CLIP) for task-level semantic guidance and micro-structural visual priors (DINOv2) for scene-aware structural information. To alleviate potential conflicts between heterogeneous priors, we devise a progressive Priors Fusion Injection (PFI) that strategically injects complementary cues at different decoder levels. Meanwhile, we equip the backbone network with an elaborate Hierarchical Mamba Module (HMM) to facilitate robust feature representation, featuring a Fourier-enhanced dual-path design that concurrently addresses global context modeling and local detail recovery. Comprehensive experiments demonstrate MPHM's state-of-the-art performance, achieving a 0.57 dB PSNR gain on the Rain200H dataset while delivering superior generalization on real-world rainy scenarios.