PiMPeR: Piecewise Dense 3D Reconstruction from Multi-View and Multi-Illumination Images

TL;DR

This paper introduces PiMPeR, a novel piecewise framework for dense 3D reconstruction from multi-view, multi-illumination images with wide baselines and uncalibrated cameras, effectively handling lighting variations and large-scale data.

Contribution

It presents the first unconstrained multi-view 3D reconstruction method that explicitly models illumination changes and scales well to large datasets.

Findings

Outperforms existing methods on benchmark datasets.

Effectively handles severe missing data and large-scale optimization.

Produces detailed 3D reconstructions with varying lighting conditions.

Abstract

In this paper, we address the problem of dense 3D reconstruction from multiple view images subject to strong lighting variations. In this regard, a new piecewise framework is proposed to explicitly take into account the change of illumination across several wide-baseline images. Unlike multi-view stereo and multi-view photometric stereo methods, this pipeline deals with wide-baseline images that are uncalibrated, in terms of both camera parameters and lighting conditions. Such a scenario is meant to avoid use of any specific imaging setup and provide a tool for normal users without any expertise. To the best of our knowledge, this paper presents the first work that deals with such unconstrained setting. We propose a coarse-to-fine approach, in which a coarse mesh is first created using a set of geometric constraints and, then, fine details are recovered by exploiting photometric properties of the scene. Augmenting the fine details on the coarse mesh is done via a final optimization step. Note that the method does not provide a generic solution for multi-view photometric stereo problem but it relaxes several common assumptions of this problem. The approach scales very well in size given its piecewise nature, dealing with large scale optimization and with severe missing data. Experiments on a benchmark dataset Robot data-set show the method performance against 3D ground truth.