Deshadow-Anything: When Segment Anything Model Meets Zero-shot shadow removal

TL;DR

This paper introduces Deshadow-Anything, a method that combines the Segment Anything Model with diffusion techniques and novel guidance strategies to perform zero-shot shadow removal while preserving image details.

Contribution

It presents a new approach integrating SAM with diffusion models and innovative guidance methods for effective zero-shot shadow removal.

Findings

Effective shadow removal with preserved image details

Improved training speed via MSAG and DDPM-AIP

Superior performance on shadow removal benchmarks

Abstract

Segment Anything (SAM), an advanced universal image segmentation model trained on an expansive visual dataset, has set a new benchmark in image segmentation and computer vision. However, it faced challenges when it came to distinguishing between shadows and their backgrounds. To address this, we developed Deshadow-Anything, considering the generalization of large-scale datasets, and we performed Fine-tuning on large-scale datasets to achieve image shadow removal. The diffusion model can diffuse along the edges and textures of an image, helping to remove shadows while preserving the details of the image. Furthermore, we design Multi-Self-Attention Guidance (MSAG) and adaptive input perturbation (DDPM-AIP) to accelerate the iterative training speed of diffusion. Experiments on shadow removal tasks demonstrate that these methods can effectively improve image restoration performance.