Polarimetric Multi-View Inverse Rendering

TL;DR

This paper introduces Polarimetric MVIR, a novel 3D reconstruction method that leverages multi-view polarization data to improve shape accuracy without relying on specific material or lighting assumptions.

Contribution

It proposes a new inverse rendering approach that integrates geometric, photometric, and polarimetric cues, including a novel cost function considering polarization ambiguities and reliability.

Findings

Accurate 3D shape reconstruction from polarization data.

Effective handling of polarization ambiguities in normal estimation.

Robust reconstruction across different materials and lighting conditions.

Abstract

A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) and the degree of polarization (DoP) of reflected light are related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color-polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric rendering errors and polarimetric errors using multi-view RGB, AoP, and DoP images, where we propose a novel polarimetric cost function that enables an effective constraint on the estimated surface normal of each vertex, while considering four possible ambiguous azimuth angles revealed from the AoP measurement. The weight for the polarimetric cost is effectively determined based on the DoP measurement, which is regarded as the reliability of polarimetric information. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific surface material and lighting condition.