Tensors, Differential Geometry and Statistical Shading Analysis

TL;DR

This paper introduces a tensor-based linear algebraic framework for shape-from-shading, analyzing invariance to illumination, developing polynomial surface solutions, and validating the advantage of using image derivatives for more robust shape reconstruction.

Contribution

It presents a novel tensor framework for shape-from-shading, enabling analysis of illumination invariance and the development of derivative-based reconstruction algorithms.

Findings

Image derivatives exhibit invariance to changing illumination.

Gradient-based shape-from-shading yields more accurate reconstructions under illumination errors.

The framework allows polynomial surface solutions consistent with observed image patches.

Abstract

We develop a linear algebraic framework for the shape-from-shading problem, because tensors arise when scalar (e.g. image) and vector (e.g. surface normal) fields are differentiated multiple times. Using this framework, we first investigate when image derivatives exhibit invariance to changing illumination by calculating the statistics of image derivatives under general distributions on the light source. Second, we apply that framework to develop Taylor-like expansions, and build a boot-strapping algorithm to find the polynomial surface solutions (under any light source) consistent with a given patch to arbitrary order. A generic constraint on the light source restricts these solutions to a 2-D subspace, plus an unknown rotation matrix. It is this unknown matrix that encapsulates the ambiguity in the problem. Finally, we use the framework to computationally validate the hypothesis that image orientations (derivatives) provide increased invariance to illumination by showing (for a Lambertian model) that a shape-from-shading algorithm matching gradients instead of intensities provides more accurate reconstructions when illumination is incorrectly estimated under a flatness prior.