Learning from Synthetic Shadows for Shadow Detection and Removal

TL;DR

This paper introduces SynShadow, a large-scale synthetic dataset for shadow detection and removal, enabling improved model training and fine-tuning for diverse shadow scenarios in computer vision.

Contribution

The creation of SynShadow, a novel synthetic dataset with a synthesis pipeline based on a physically-grounded model, addressing data scarcity in shadow removal tasks.

Findings

Models trained on SynShadow perform well on challenging benchmarks.

Fine-tuning from SynShadow improves existing shadow detection and removal models.

SynShadow enables better generalization to diverse shadow shapes and intensities.

Abstract

Shadow removal is an essential task in computer vision and computer graphics. Recent shadow removal approaches all train convolutional neural networks (CNN) on real paired shadow/shadow-free or shadow/shadow-free/mask image datasets. However, obtaining a large-scale, diverse, and accurate dataset has been a big challenge, and it limits the performance of the learned models on shadow images with unseen shapes/intensities. To overcome this challenge, we present SynShadow, a novel large-scale synthetic shadow/shadow-free/matte image triplets dataset and a pipeline to synthesize it. We extend a physically-grounded shadow illumination model and synthesize a shadow image given an arbitrary combination of a shadow-free image, a matte image, and shadow attenuation parameters. Owing to the diversity, quantity, and quality of SynShadow, we demonstrate that shadow removal models trained on SynShadow perform well in removing shadows with diverse shapes and intensities on some challenging benchmarks. Furthermore, we show that merely fine-tuning from a SynShadow-pre-trained model improves existing shadow detection and removal models. Codes are publicly available at https://github.com/naoto0804/SynShadow.