TL;DR
LitAR is a novel framework for mobile augmented reality that provides realistic, visually-coherent environment lighting by combining spatial variance-aware reconstruction, noise-tolerant data capturing, and real-time rendering techniques.
Contribution
LitAR introduces a comprehensive lighting reconstruction framework that improves visual coherence and efficiency for mobile AR by addressing spatial variance, data quality, and real-time rendering challenges.
Findings
Enables reflective rendering with correct color tones in mobile AR.
Uses two-field lighting reconstruction for spatial variance handling.
Employs real-time environment map rendering techniques to improve efficiency.
Abstract
An accurate understanding of omnidirectional environment lighting is crucial for high-quality virtual object rendering in mobile augmented reality (AR). In particular, to support reflective rendering, existing methods have leveraged deep learning models to estimate or have used physical light probes to capture physical lighting, typically represented in the form of an environment map. However, these methods often fail to provide visually coherent details or require additional setups. For example, the commercial framework ARKit uses a convolutional neural network that can generate realistic environment maps; however the corresponding reflective rendering might not match the physical environments. In this work, we present the design and implementation of a lighting reconstruction framework called LitAR that enables realistic and visually-coherent rendering. LitAR addresses several…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
Methodsfail
