Shadow Art Revisited: A Differentiable Rendering Based Approach

TL;DR

This paper introduces a differentiable rendering framework to reconstruct complex 3D sculptures from shadow images, enabling artistic shadow art creation without relying on 3D ground truth data.

Contribution

It presents a novel use of differentiable rendering for shadow art, allowing 3D shape optimization solely from shadow images, expanding artistic and reconstruction possibilities.

Findings

Successfully generates complex 3D sculptures from shadow images.

Demonstrates application to faces, characters, and sketch-based reconstruction.

Outperforms traditional methods in shadow art generation.

Abstract

While recent learning based methods have been observed to be superior for several vision-related applications, their potential in generating artistic effects has not been explored much. One such interesting application is Shadow Art - a unique form of sculptural art where 2D shadows cast by a 3D sculpture produce artistic effects. In this work, we revisit shadow art using differentiable rendering based optimization frameworks to obtain the 3D sculpture from a set of shadow (binary) images and their corresponding projection information. Specifically, we discuss shape optimization through voxel as well as mesh-based differentiable renderers. Our choice of using differentiable rendering for generating shadow art sculptures can be attributed to its ability to learn the underlying 3D geometry solely from image data, thus reducing the dependence on 3D ground truth. The qualitative and quantitative results demonstrate the potential of the proposed framework in generating complex 3D sculptures that go beyond those seen in contemporary art pieces using just a set of shadow images as input. Further, we demonstrate the generation of 3D sculptures to cast shadows of faces, animated movie characters, and applicability of the framework to sketch-based 3D reconstruction of underlying shapes.