Relightable Gaussian Codec Avatars
Shunsuke Saito, Gabriel Schwartz, Tomas Simon, Junxuan Li, Giljoo Nam

TL;DR
This paper introduces a high-fidelity, relightable head avatar model using 3D Gaussian geometry and a novel appearance model, enabling real-time rendering and animation with detailed features and diverse lighting conditions.
Contribution
The work presents a new Gaussian-based geometry and a learnable radiance transfer appearance model for realistic, relightable avatars capable of real-time performance and detailed expression animation.
Findings
Captures sub-millimeter details like hair strands and pores.
Achieves real-time relighting with all-frequency reflections.
Outperforms existing methods in fidelity and speed.
Abstract
The fidelity of relighting is bounded by both geometry and appearance representations. For geometry, both mesh and volumetric approaches have difficulty modeling intricate structures like 3D hair geometry. For appearance, existing relighting models are limited in fidelity and often too slow to render in real-time with high-resolution continuous environments. In this work, we present Relightable Gaussian Codec Avatars, a method to build high-fidelity relightable head avatars that can be animated to generate novel expressions. Our geometry model based on 3D Gaussians can capture 3D-consistent sub-millimeter details such as hair strands and pores on dynamic face sequences. To support diverse materials of human heads such as the eyes, skin, and hair in a unified manner, we present a novel relightable appearance model based on learnable radiance transfer. Together with global…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsFace recognition and analysis · 3D Shape Modeling and Analysis · Advanced Vision and Imaging
