No Shadow Left Behind: Removing Objects and their Shadows using Approximate Lighting and Geometry

TL;DR

This paper presents a deep learning method for removing objects and their shadows from images by leveraging approximate scene models, enabling effective shadow removal across diverse scenarios.

Contribution

The authors introduce a novel deep learning pipeline that removes objects and their shadows simultaneously using approximate scene models, improving over existing inpainting methods.

Findings

Effective removal of various types of shadows from images.

The pipeline performs well on both synthetic and real scenes.

Outperforms traditional inpainting methods in shadow removal quality.

Abstract

Removing objects from images is a challenging problem that is important for many applications, including mixed reality. For believable results, the shadows that the object casts should also be removed. Current inpainting-based methods only remove the object itself, leaving shadows behind, or at best require specifying shadow regions to inpaint. We introduce a deep learning pipeline for removing a shadow along with its caster. We leverage rough scene models in order to remove a wide variety of shadows (hard or soft, dark or subtle, large or thin) from surfaces with a wide variety of textures. We train our pipeline on synthetically rendered data, and show qualitative and quantitative results on both synthetic and real scenes.