Half-body Portrait Relighting with Overcomplete Lighting Representation
Guoxian Song, Tat-Jen Cham, Jianfei Cai, Jianmin Zheng

TL;DR
This paper introduces a neural relighting method for half-body portraits using an overcomplete lighting representation and a novel neural rendering technique, enabling high-quality relighting and lighting interpolation.
Contribution
It proposes an overcomplete lighting representation and a multiplicative neural render for improved portrait relighting, along with a large-scale dataset for training.
Findings
Outperforms existing referral-based relighting methods
Enables smooth lighting interpolation and rotations
Generalizes well to real images
Abstract
We present a neural-based model for relighting a half-body portrait image by simply referring to another portrait image with the desired lighting condition. Rather than following classical inverse rendering methodology that involves estimating normals, albedo and environment maps, we implicitly encode the subject and lighting in a latent space, and use these latent codes to generate relighted images by neural rendering. A key technical innovation is the use of a novel overcomplete lighting representation, which facilitates lighting interpolation in the latent space, as well as helping regularize the self-organization of the lighting latent space during training. In addition, we propose a novel multiplicative neural render that more effectively combines the subject and lighting latent codes for rendering. We also created a large-scale photorealistic rendered relighting dataset for…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
