Real-Time Depth Refinement for Specular Objects

TL;DR

This paper introduces a real-time shape from shading method that improves depth refinement for both diffuse and specular objects using RGB-D scanner IR data, achieving state-of-the-art results.

Contribution

A novel lighting model and shape from shading framework that effectively handles specularities in depth refinement, enabling real-time processing.

Findings

Produces high-accuracy depth maps for specular objects

Operates in real-time using RGB-D scanner IR data

Outperforms existing depth refinement methods

Abstract

The introduction of consumer RGB-D scanners set off a major boost in 3D computer vision research. Yet, the precision of existing depth scanners is not accurate enough to recover fine details of a scanned object. While modern shading based depth refinement methods have been proven to work well with Lambertian objects, they break down in the presence of specularities. We present a novel shape from shading framework that addresses this issue and enhances both diffuse and specular objects' depth profiles. We take advantage of the built-in monochromatic IR projector and IR images of the RGB-D scanners and present a lighting model that accounts for the specular regions in the input image. Using this model, we reconstruct the depth map in real-time. Both quantitative tests and visual evaluations prove that the proposed method produces state of the art depth reconstruction results.