TL;DR
This paper presents novel photometric depth super-resolution methods that enhance low-resolution depth maps using shape-from-shading and photometric stereo, employing variational, neural, and multi-shot strategies for improved accuracy.
Contribution
It introduces a variational approach for depth upsampling, relaxes reflectance assumptions with neural networks, and discusses a multi-shot method requiring no prior reflectance knowledge.
Findings
Effective depth upsampling on synthetic and real data
Neural network-based reflectance estimation improves results
Multi-shot approach achieves high accuracy without prior reflectance
Abstract
This study explores the use of photometric techniques (shape-from-shading and uncalibrated photometric stereo) for upsampling the low-resolution depth map from an RGB-D sensor to the higher resolution of the companion RGB image. A single-shot variational approach is first put forward, which is effective as long as the target's reflectance is piecewise-constant. It is then shown that this dependency upon a specific reflectance model can be relaxed by focusing on a specific class of objects (e.g., faces), and delegate reflectance estimation to a deep neural network. A multi-shot strategy based on randomly varying lighting conditions is eventually discussed. It requires no training or prior on the reflectance, yet this comes at the price of a dedicated acquisition setup. Both quantitative and qualitative evaluations illustrate the effectiveness of the proposed methods on synthetic and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
