Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Video

TL;DR

SGIA introduces a fast, relightable, and accurate surfel-based method for dynamic human avatar reconstruction from monocular video, enabling realistic lighting manipulation and pose changes with improved efficiency.

Contribution

The paper presents SGIA, a novel surfel-based Gaussian inverse rendering approach that models PBR properties for dynamic humans, enhancing relighting and geometry reconstruction speed and accuracy.

Findings

Achieves highly accurate physical property estimation.

Enables realistic relighting of dynamic avatars.

Significantly faster rendering compared to implicit methods.

Abstract

Efficient and accurate reconstruction of a relightable, dynamic clothed human avatar from a monocular video is crucial for the entertainment industry. This paper presents SGIA (Surfel-based Gaussian Inverse Avatar), which introduces efficient training and rendering for relightable dynamic human reconstruction. SGIA advances previous Gaussian Avatar methods by comprehensively modeling Physically-Based Rendering (PBR) properties for clothed human avatars, allowing for the manipulation of avatars into novel poses under diverse lighting conditions. Specifically, our approach integrates pre-integration and image-based lighting for fast light calculations that surpass the performance of existing implicit-based techniques. To address challenges related to material lighting disentanglement and accurate geometry reconstruction, we propose an innovative occlusion approximation strategy and a progressive training approach. Extensive experiments demonstrate that SGIA not only achieves highly accurate physical properties but also significantly enhances the realistic relighting of dynamic human avatars, providing a substantial speed advantage. We exhibit more results in our project page: https://GS-IA.github.io.