# Detailed Surface Geometry and Albedo Recovery from RGB-D Video Under Natural Illumination

## TL;DR

This paper introduces a method to enhance depth maps from RGB-D videos by leveraging natural lighting variations and shading cues, enabling detailed surface and albedo recovery without controlled conditions.

## Contribution

It proposes a novel approach that uses natural illumination changes and a robust pixel matching technique to improve surface detail and intrinsic property recovery from RGB-D videos.

## Key findings

- Outperforms optical flow in lighting variation scenarios
- Successfully recovers surface normals and albedo without regularization
- Validated on synthetic and real datasets showing superior detail recovery

## Abstract

In this paper we present a novel approach for depth map enhancement from an RGB-D video sequence. The basic idea is to exploit the shading information in the color image. Instead of making assumption about surface albedo or controlled object motion and lighting, we use the lighting variations introduced by casual object movement. We are effectively calculating photometric stereo from a moving object under natural illuminations. The key technical challenge is to establish correspondences over the entire image set. We therefore develop a lighting insensitive robust pixel matching technique that out-performs optical flow method in presence of lighting variations. In addition we present an expectation-maximization framework to recover the surface normal and albedo simultaneously, without any regularization term. We have validated our method on both synthetic and real datasets to show its superior performance on both surface details recovery and intrinsic decomposition.