Lite2Relight: 3D-aware Single Image Portrait Relighting
Pramod Rao, Gereon Fox, Abhimitra Meka, Mallikarjun B R, Fangneng, Zhan, Tim Weyrich, Bernd Bickel, Hanspeter Pfister, Wojciech Matusik, Mohamed, Elgharib, Christian Theobalt

TL;DR
Lite2Relight is a novel method that enables photorealistic, 3D-consistent portrait relighting and view synthesis from a single image at interactive speeds, improving over prior methods in realism and generalization.
Contribution
It introduces a new approach extending EG3D with a lightstage dataset and a geometry-aware encoder for high-quality, 3D-consistent relighting from a single image.
Findings
Outperforms state-of-the-art in realism and efficacy
Produces 3D-consistent results including hair and expressions
Operates at interactive speeds for practical applications
Abstract
Achieving photorealistic 3D view synthesis and relighting of human portraits is pivotal for advancing AR/VR applications. Existing methodologies in portrait relighting demonstrate substantial limitations in terms of generalization and 3D consistency, coupled with inaccuracies in physically realistic lighting and identity preservation. Furthermore, personalization from a single view is difficult to achieve and often requires multiview images during the testing phase or involves slow optimization processes. This paper introduces Lite2Relight, a novel technique that can predict 3D consistent head poses of portraits while performing physically plausible light editing at interactive speed. Our method uniquely extends the generative capabilities and efficient volumetric representation of EG3D, leveraging a lightstage dataset to implicitly disentangle face reflectance and perform relighting…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
