Shape, Illumination, and Reflectance from Shading

TL;DR

This paper introduces a statistical inference approach to recover 3D shape, reflectance, and illumination from a single image, outperforming previous methods by leveraging natural priors and optimization.

Contribution

It presents a unified framework that combines multiple intrinsic image problems into a single optimization for single-image analysis, improving accuracy.

Findings

Outperforms previous solutions in shape-from-shading and intrinsic image tasks

Effectively infers 3D shape, reflectance, and illumination from one image

Utilizes natural priors to constrain the ill-posed problem

Abstract

A fundamental problem in computer vision is that of inferring the intrinsic, 3D structure of the world from flat, 2D images of that world. Traditional methods for recovering scene properties such as shape, reflectance, or illumination rely on multiple observations of the same scene to overconstrain the problem. Recovering these same properties from a single image seems almost impossible in comparison -- there are an infinite number of shapes, paint, and lights that exactly reproduce a single image. However, certain explanations are more likely than others: surfaces tend to be smooth, paint tends to be uniform, and illumination tends to be natural. We therefore pose this problem as one of statistical inference, and define an optimization problem that searches for the *most likely* explanation of a single image. Our technique can be viewed as a superset of several classic computer vision problems (shape-from-shading, intrinsic images, color constancy, illumination estimation, etc) and outperforms all previous solutions to those constituent problems.