PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularization

TL;DR

This paper introduces a neural inverse rendering method that jointly optimizes lighting, models indirect illumination, and regularizes surface reflectance to improve accuracy in reflectance and illumination decomposition from images.

Contribution

The proposed approach jointly optimizes light source position, models indirect illumination with differentiable rendering, and introduces a DINO feature-based regularization for better surface reflectance decomposition.

Findings

Outperforms state-of-the-art in reflectance decomposition

Effectively models self-shadows and inter-reflections

Achieves accurate material and illumination estimation

Abstract

This paper addresses the problem of inverse rendering from photometric images. Existing approaches for this problem suffer from the effects of self-shadows, inter-reflections, and lack of constraints on the surface reflectance, leading to inaccurate decomposition of reflectance and illumination due to the ill-posed nature of inverse rendering. In this work, we propose a new method for neural inverse rendering. Our method jointly optimizes the light source position to account for the self-shadows in images, and computes indirect illumination using a differentiable rendering layer and an importance sampling strategy. To enhance surface reflectance decomposition, we introduce a new regularization by distilling DINO features to foster accurate and consistent material decomposition. Extensive experiments on synthetic and real datasets demonstrate that our method outperforms the state-of-the-art methods in reflectance decomposition.