Spectral-Structured Diffusion for Single-Image Rain Removal

TL;DR

SpectralDiff is a novel spectral-structured diffusion framework that effectively removes rain streaks from single images by leveraging spectral perturbations and a full-product U-Net architecture, achieving competitive results with improved efficiency.

Contribution

The paper introduces SpectralDiff, a spectral-structured diffusion model with a full-product U-Net architecture for efficient and effective single-image rain removal.

Findings

Achieves competitive rain removal performance on benchmarks.

Improves computational efficiency with the full-product U-Net.

Demonstrates effectiveness on synthetic and real-world images.

Abstract

Rain streaks manifest as directional and frequency-concentrated structures that overlap across multiple scales, making single-image rain removal particularly challenging. While diffusion-based restoration models provide a powerful framework for progressive denoising, standard spatial-domain diffusion does not explicitly account for such structured spectral characteristics. We introduce SpectralDiff, a spectral-structured diffusion-based framework tailored for single-image rain removal. Rather than redefining the diffusion formulation, our method incorporates structured spectral perturbations to guide the progressive suppression of multi-directional rain components. To support this design, we further propose a full-product U-Net architecture that leverages the convolution theorem to replace convolution operations with element-wise product layers, improving computational efficiency while preserving modeling capacity. Extensive experiments on synthetic and real-world benchmarks demonstrate that SpectralDiff achieves competitive rain removal performance with improved model compactness and favorable inference efficiency compared to existing diffusion-based approaches.