Diff-Shadow: Global-guided Diffusion Model for Shadow Removal

TL;DR

Diff-Shadow introduces a global-guided diffusion model with a parallel UNet architecture and novel modules to improve shadow removal, achieving superior image quality and illumination consistency over previous methods.

Contribution

The paper presents a novel parallel UNet diffusion framework with Reweight Cross Attention and Global-guided Sampling Strategy for enhanced shadow removal.

Findings

Significant PSNR improvement on ISTD dataset from 32.33dB to 33.69dB.

Effective integration of global guidance improves shadow boundary recovery.

Outperforms state-of-the-art methods in shadow removal quality.

Abstract

We propose Diff-Shadow, a global-guided diffusion model for shadow removal. Previous transformer-based approaches can utilize global information to relate shadow and non-shadow regions but are limited in their synthesis ability and recover images with obvious boundaries. In contrast, diffusion-based methods can generate better content but they are not exempt from issues related to inconsistent illumination. In this work, we combine the advantages of diffusion models and global guidance to achieve shadow-free restoration. Specifically, we propose a parallel UNets architecture: 1) the local branch performs the patch-based noise estimation in the diffusion process, and 2) the global branch recovers the low-resolution shadow-free images. A Reweight Cross Attention (RCA) module is designed to integrate global contextual information of non-shadow regions into the local branch. We further design a Global-guided Sampling Strategy (GSS) that mitigates patch boundary issues and ensures consistent illumination across shaded and unshaded regions in the recovered image. Comprehensive experiments on datasets ISTD, ISTD+, and SRD have demonstrated the effectiveness of Diff-Shadow. Compared to state-of-the-art methods, our method achieves a significant improvement in terms of PSNR, increasing from 32.33dB to 33.69dB on the ISTD dataset.