Learning Efficient Photometric Feature Transform for Multi-view Stereo

TL;DR

This paper introduces a learnable photometric feature transform that enhances multi-view stereo 3D reconstruction by jointly optimizing illumination and feature extraction in a differentiable framework, demonstrating superior results on challenging data.

Contribution

A novel differentiable framework for learning photometric features that adapt to various input data for improved multi-view stereo reconstruction.

Findings

Achieves high-quality 3D reconstructions on challenging objects.

Outperforms state-of-the-art techniques in multi-view stereo.

Effectively adapts to different illumination conditions.

Abstract

We present a novel framework to learn to convert the perpixel photometric information at each view into spatially distinctive and view-invariant low-level features, which can be plugged into existing multi-view stereo pipeline for enhanced 3D reconstruction. Both the illumination conditions during acquisition and the subsequent per-pixel feature transform can be jointly optimized in a differentiable fashion. Our framework automatically adapts to and makes efficient use of the geometric information available in different forms of input data. High-quality 3D reconstructions of a variety of challenging objects are demonstrated on the data captured with an illumination multiplexing device, as well as a point light. Our results compare favorably with state-of-the-art techniques.