What Is Around The Camera?
Stamatios Georgoulis, Konstantinos Rematas, Tobias Ritschel, Mario, Fritz, Tinne Tuytelaars, Luc Van Gool

TL;DR
This paper explores how much environmental information can be inferred from a foreground object and background in an image, using a learning-based approach that models environment and material properties from reflectance maps.
Contribution
It introduces a novel method to predict environment details from images of objects with complex reflectance, combining environment and material modeling from synthesized data.
Findings
Objects with multiple materials provide more environmental information.
The method performs well on real-world data despite training on synthetic data.
Reflectance maps effectively encode environmental cues for inference.
Abstract
How much does a single image reveal about the environment it was taken in? In this paper, we investigate how much of that information can be retrieved from a foreground object, combined with the background (i.e. the visible part of the environment). Assuming it is not perfectly diffuse, the foreground object acts as a complexly shaped and far-from-perfect mirror. An additional challenge is that its appearance confounds the light coming from the environment with the unknown materials it is made of. We propose a learning-based approach to predict the environment from multiple reflectance maps that are computed from approximate surface normals. The proposed method allows us to jointly model the statistics of environments and material properties. We train our system from synthesized training data, but demonstrate its applicability to real-world data. Interestingly, our analysis shows that…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Image Enhancement Techniques
