Inferring Super-Resolution Depth from a Moving Light-Source Enhanced RGB-D Sensor: A Variational Approach

TL;DR

This paper introduces a variational method that leverages a moving LED light source and multi-view RGB-D data to enhance depth map resolution without requiring calibration, achieving high-quality 3D reconstructions.

Contribution

It presents a novel joint optimization framework for super-resolution depth estimation using uncalibrated photometric stereo with a moving light source.

Findings

Effective depth super-resolution on synthetic datasets

High-quality depth and reflectance recovery on real-world data

No calibration needed for lighting or camera motion

Abstract

A novel approach towards depth map super-resolution using multi-view uncalibrated photometric stereo is presented. Practically, an LED light source is attached to a commodity RGB-D sensor and is used to capture objects from multiple viewpoints with unknown motion. This non-static camera-to-object setup is described with a nonconvex variational approach such that no calibration on lighting or camera motion is required due to the formulation of an end-to-end joint optimization problem. Solving the proposed variational model results in high resolution depth, reflectance and camera pose estimates, as we show on challenging synthetic and real-world datasets.