Polarimetric Multi-View Inverse Rendering

TL;DR

This paper introduces Polarimetric MVIR, a 3D reconstruction method that leverages multi-view polarization images to improve surface normal estimation and shape accuracy without material assumptions.

Contribution

It proposes a novel polarimetric rendering cost function that effectively constrains surface normals considering polarization ambiguities, enhancing 3D reconstruction accuracy.

Findings

Successfully reconstructs detailed 3D shapes from synthetic and real data.

Does not rely on material-specific polarized reflection assumptions.

Abstract

A polarization camera has great potential for 3D reconstruction since the angle of polarization (AoP) of reflected light is related to an object's surface normal. In this paper, we propose a novel 3D reconstruction method called Polarimetric Multi-View Inverse Rendering (Polarimetric MVIR) that effectively exploits geometric, photometric, and polarimetric cues extracted from input multi-view color polarization images. We first estimate camera poses and an initial 3D model by geometric reconstruction with a standard structure-from-motion and multi-view stereo pipeline. We then refine the initial model by optimizing photometric and polarimetric rendering errors using multi-view RGB and AoP images, where we propose a novel polarimetric rendering cost function that enables us to effectively constrain each estimated surface vertex's normal while considering four possible ambiguous azimuth angles revealed from the AoP measurement. Experimental results using both synthetic and real data demonstrate that our Polarimetric MVIR can reconstruct a detailed 3D shape without assuming a specific polarized reflection depending on the material.