Single-image Full-body Human Relighting
Manuel Lagunas, Xin Sun, Jimei Yang, Ruben Villegas, Jianming Zhang,, Zhixin Shu, Belen Masia, and Diego Gutierrez

TL;DR
This paper introduces a novel data-driven method for relighting full-body human images from a single photo, utilizing scene decomposition with PRT and spherical harmonics, and explicitly modeling reflectance properties.
Contribution
It presents a new deep learning architecture that improves full-body human relighting by explicitly modeling diffuse and specular reflectance and incorporating a residual error term.
Findings
Outperforms previous methods on synthetic and real images
Effectively models diffuse and specular reflectance
Achieves more realistic relighting results
Abstract
We present a single-image data-driven method to automatically relight images with full-body humans in them. Our framework is based on a realistic scene decomposition leveraging precomputed radiance transfer (PRT) and spherical harmonics (SH) lighting. In contrast to previous work, we lift the assumptions on Lambertian materials and explicitly model diffuse and specular reflectance in our data. Moreover, we introduce an additional light-dependent residual term that accounts for errors in the PRT-based image reconstruction. We propose a new deep learning architecture, tailored to the decomposition performed in PRT, that is trained using a combination of L1, logarithmic, and rendering losses. Our model outperforms the state of the art for full-body human relighting both with synthetic images and photographs.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
