SSN: Soft Shadow Network for Image Compositing

TL;DR

The paper presents SSN, an interactive neural network that generates controllable, realistic soft shadows for image compositing in real-time, using user inputs and environment maps.

Contribution

It introduces a novel Soft Shadow Network with an ambient occlusion prediction module and inverse shadow map representation, enabling real-time, controllable shadow generation for diverse images.

Findings

Produces realistic soft shadows comparable to physics-based renderers

Operates in real-time for interactive applications

Allows user control over shadow characteristics

Abstract

We introduce an interactive Soft Shadow Network (SSN) to generates controllable soft shadows for image compositing. SSN takes a 2D object mask as input and thus is agnostic to image types such as painting and vector art. An environment light map is used to control the shadow's characteristics, such as angle and softness. SSN employs an Ambient Occlusion Prediction module to predict an intermediate ambient occlusion map, which can be further refined by the user to provides geometric cues to modulate the shadow generation. To train our model, we design an efficient pipeline to produce diverse soft shadow training data using 3D object models. In addition, we propose an inverse shadow map representation to improve model training. We demonstrate that our model produces realistic soft shadows in real-time. Our user studies show that the generated shadows are often indistinguishable from shadows calculated by a physics-based renderer and users can easily use SSN through an interactive application to generate specific shadow effects in minutes.