SupeRVol: Super-Resolution Shape and Reflectance Estimation in Inverse Volume Rendering

TL;DR

SupeRVol is an end-to-end inverse rendering pipeline that reconstructs high-resolution 3D shape and material properties from low-resolution color images using differentiable volume rendering and physically based illumination models.

Contribution

It introduces a novel super-resolution inverse rendering method that jointly estimates shape and reflectance from images, outperforming previous approaches in quality.

Findings

Achieves state-of-the-art inverse rendering quality.

Produces sharper reconstructions than input images.

Effective in 3D modeling from low-resolution images.

Abstract

We propose an end-to-end inverse rendering pipeline called SupeRVol that allows us to recover 3D shape and material parameters from a set of color images in a super-resolution manner. To this end, we represent both the bidirectional reflectance distribution function (BRDF) and the signed distance function (SDF) by multi-layer perceptrons. In order to obtain both the surface shape and its reflectance properties, we revert to a differentiable volume renderer with a physically based illumination model that allows us to decouple reflectance and lighting. This physical model takes into account the effect of the camera's point spread function thereby enabling a reconstruction of shape and material in a super-resolution quality. Experimental validation confirms that SupeRVol achieves state of the art performance in terms of inverse rendering quality. It generates reconstructions that are sharper than the individual input images, making this method ideally suited for 3D modeling from low-resolution imagery.