MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis

TL;DR

MetaShadow is a versatile framework that advances object-centered shadow detection, removal, and synthesis, significantly improving realism and control in image editing tasks.

Contribution

It introduces a novel integrated approach combining detection, removal, and synthesis with optimized feature learning for realistic shadow generation.

Findings

Outperforms state-of-the-art methods on multiple benchmarks

Enhances realism in object removal, relocation, and insertion tasks

Achieves significant improvements in shadow detection, removal, and synthesis

Abstract

Shadows are often under-considered or even ignored in image editing applications, limiting the realism of the edited results. In this paper, we introduce MetaShadow, a three-in-one versatile framework that enables detection, removal, and controllable synthesis of shadows in natural images in an object-centered fashion. MetaShadow combines the strengths of two cooperative components: Shadow Analyzer, for object-centered shadow detection and removal, and Shadow Synthesizer, for reference-based controllable shadow synthesis. Notably, we optimize the learning of the intermediate features from Shadow Analyzer to guide Shadow Synthesizer to generate more realistic shadows that blend seamlessly with the scene. Extensive evaluations on multiple shadow benchmark datasets show significant improvements of MetaShadow over the existing state-of-the-art methods on object-centered shadow detection, removal, and synthesis. MetaShadow excels in image-editing tasks such as object removal, relocation, and insertion, pushing the boundaries of object-centered image editing.