Learning Photometric Feature Transform for Free-form Object Scan

TL;DR

This paper introduces a learning-based framework that transforms photometric data from multiple views into features that improve 3D reconstruction quality, enabling effective geometry and reflectance capture from simple handheld setups.

Contribution

A novel system that jointly learns photometric feature transformation and aggregation, enhancing multi-view stereo for free-form object scanning from unstructured views.

Findings

Achieves high-quality 3D reconstructions comparable to professional scanners.

Demonstrates effective capture of geometry and anisotropic reflectance.

Operates with a lightweight, portable hardware setup.

Abstract

We propose a novel framework to automatically learn to aggregate and transform photometric measurements from multiple unstructured views into spatially distinctive and view-invariant low-level features, which are subsequently fed to a multi-view stereo pipeline to enhance 3D reconstruction. The illumination conditions during acquisition and the feature transform are jointly trained on a large amount of synthetic data. We further build a system to reconstruct both the geometry and anisotropic reflectance of a variety of challenging objects from hand-held scans. The effectiveness of the system is demonstrated with a lightweight prototype, consisting of a camera and an array of LEDs, as well as an off-the-shelf tablet. Our results are validated against reconstructions from a professional 3D scanner and photographs, and compare favorably with state-of-the-art techniques.