Towards Geometry Guided Neural Relighting with Flash Photography
Di Qiu, Jin Zeng, Zhanghan Ke, Wenxiu Sun, Chengxi Yang

TL;DR
This paper introduces a deep learning framework that uses a single flash photograph and its depth map to achieve realistic image relighting with high-frequency effects, simplifying data capture and improving over existing methods.
Contribution
It leverages geometric information from depth maps to enhance single-image relighting, a novel approach compared to prior multi-image techniques.
Findings
Outperforms state-of-the-art in intrinsic image decomposition
Achieves realistic high-frequency effects in relighting
Demonstrates effectiveness on real mobile phone photos
Abstract
Previous image based relighting methods require capturing multiple images to acquire high frequency lighting effect under different lighting conditions, which needs nontrivial effort and may be unrealistic in certain practical use scenarios. While such approaches rely entirely on cleverly sampling the color images under different lighting conditions, little has been done to utilize geometric information that crucially influences the high-frequency features in the images, such as glossy highlight and cast shadow. We therefore propose a framework for image relighting from a single flash photograph with its corresponding depth map using deep learning. By incorporating the depth map, our approach is able to extrapolate realistic high-frequency effects under novel lighting via geometry guided image decomposition from the flashlight image, and predict the cast shadow map from the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Image Enhancement Techniques · Computer Graphics and Visualization Techniques
