SIRfyN: Single Image Relighting from your Neighbors
D.A. Forsyth, Anand Bhattad, Pranav Asthana, Yuanyi Zhong, Yuxiong, Wang

TL;DR
This paper introduces SIRfyN, a novel method for relighting a scene in a single image by estimating the illumination cone using similar scenes, without requiring ground truth data, enabling natural scene relighting and data augmentation.
Contribution
The paper presents a new theory and method to estimate scene lighting from similar scenes, avoiding the need for inverse graphics datasets or ground truth, for realistic image relighting.
Findings
Effectively erases and restores indoor shadows
Can steer light around a scene
Produces images suitable for data augmentation
Abstract
We show how to relight a scene, depicted in a single image, such that (a) the overall shading has changed and (b) the resulting image looks like a natural image of that scene. Applications for such a procedure include generating training data and building authoring environments. Naive methods for doing this fail. One reason is that shading and albedo are quite strongly related; for example, sharp boundaries in shading tend to appear at depth discontinuities, which usually apparent in albedo. The same scene can be lit in different ways, and established theory shows the different lightings form a cone (the illumination cone). Novel theory shows that one can use similar scenes to estimate the different lightings that apply to a given scene, with bounded expected error. Our method exploits this theory to estimate a representation of the available lighting fields in the form of imputed…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsComputer Graphics and Visualization Techniques · Advanced Vision and Imaging · Image Enhancement Techniques
