A Dark Flash Normal Camera

TL;DR

This paper introduces a method to estimate surface normals and reflectance maps of scenes with people in uncontrolled lighting by using a single near-infrared dark flash image alongside a visible light image, improving image quality and downstream processing.

Contribution

It presents a novel approach combining visible and NIR images to recover detailed surface properties, with a new training technique leveraging stereo and shading cues.

Findings

Effective normal and reflectance estimation across diverse conditions

Improved stereo geometry and shadow filling applications

Robust performance demonstrated on various subjects

Abstract

Casual photography is often performed in uncontrolled lighting that can result in low quality images and degrade the performance of downstream processing. We consider the problem of estimating surface normal and reflectance maps of scenes depicting people despite these conditions by supplementing the available visible illumination with a single near infrared (NIR) light source and camera, a so-called "dark flash image". Our method takes as input a single color image captured under arbitrary visible lighting and a single dark flash image captured under controlled front-lit NIR lighting at the same viewpoint, and computes a normal map, a diffuse albedo map, and a specular intensity map of the scene. Since ground truth normal and reflectance maps of faces are difficult to capture, we propose a novel training technique that combines information from two readily available and complementary sources: a stereo depth signal and photometric shading cues. We evaluate our method over a range of subjects and lighting conditions and describe two applications: optimizing stereo geometry and filling the shadows in an image.