Rotated Lights for Consistent and Efficient 2D Gaussians Inverse Rendering

TL;DR

RotLight introduces a simple rotation-based capturing setup that, with minimal rotations, significantly improves the accuracy of inverse rendering by reducing artifacts and enhancing albedo estimation, applicable to both synthetic and real datasets.

Contribution

The paper proposes RotLight, a novel rotation-based capturing method with a proxy mesh to improve 2D Gaussian splatting inverse rendering accuracy and efficiency.

Findings

Effective with as few as two rotations

Achieves superior albedo estimation

Works well on synthetic and real datasets

Abstract

Inverse rendering aims to decompose a scene into its geometry, material properties and light conditions under a certain rendering model. It has wide applications like view synthesis, relighting, and scene editing. In recent years, inverse rendering methods have been inspired by view synthesis approaches like neural radiance fields and Gaussian splatting, which are capable of efficiently decomposing a scene into its geometry and radiance. They then further estimate the material and lighting that lead to the observed scene radiance. However, the latter step is highly ambiguous and prior works suffer from inaccurate color and baked shadows in their albedo estimation albeit their regularization. To this end, we propose RotLight, a simple capturing setup, to address the ambiguity. Compared to a usual capture, RotLight only requires the object to be rotated several times during the process. We show that as few as two rotations is effective in reducing artifacts. To further improve 2DGS-based inverse rendering, we additionally introduce a proxy mesh that not only allows accurate incident light tracing, but also enables a residual constraint and improves global illumination handling. We demonstrate with both synthetic and real world datasets that our method achieves superior albedo estimation while keeping efficient computation.