Shape and Material Capture at Home

TL;DR

This paper introduces RecNet, a recursive neural architecture for estimating object geometry and reflectance from minimal images using a simple capture setup, achieving high-resolution results especially in challenging regions.

Contribution

The paper presents RecNet, a novel recursive neural network that improves geometry and reflectance estimation from limited images and simple capture procedures.

Findings

More accurate surface normal and albedo estimation in specular and shadow regions.

Effective at high resolutions up to 1024x1024 during inference.

Works with three or fewer input images.

Abstract

In this paper, we present a technique for estimating the geometry and reflectance of objects using only a camera, flashlight, and optionally a tripod. We propose a simple data capture technique in which the user goes around the object, illuminating it with a flashlight and capturing only a few images. Our main technical contribution is the introduction of a recursive neural architecture, which can predict geometry and reflectance at 2^{k}*2^{k} resolution given an input image at 2^{k}*2^{k} and estimated geometry and reflectance from the previous step at 2^{k-1}*2^{k-1}. This recursive architecture, termed RecNet, is trained with 256x256 resolution but can easily operate on 1024x1024 images during inference. We show that our method produces more accurate surface normal and albedo, especially in regions of specular highlights and cast shadows, compared to previous approaches, given three or fewer input images. For the video and code, please visit the project website http://dlichy.github.io/ShapeAndMaterialAtHome/.