UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
Chih-Hao Lin, Bohan Liu, Yi-Ting Chen, Kuan-Sheng Chen, David Forsyth,, Jia-Bin Huang, Anand Bhattad, Shenlong Wang

TL;DR
UrbanIR is a novel inverse rendering framework that accurately infers scene geometry, materials, and lighting from a single wide-baseline video, enabling realistic free-viewpoint rendering and scene editing in urban environments.
Contribution
It introduces new loss functions and techniques for improved inverse graphics inference from challenging urban scene videos, outperforming existing methods in accuracy and realism.
Findings
Accurate shape, albedo, and illumination estimation from single videos.
Enhanced shadow volume and lighting control capabilities.
Superior rendering quality compared to prior state-of-the-art methods.
Abstract
We present UrbanIR (Urban Scene Inverse Rendering), a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video. It accurately infers shape, albedo, visibility, and sun and sky illumination from wide-baseline videos, such as those from car-mounted cameras, differing from NeRF's dense view settings. In this context, standard methods often yield subpar geometry and material estimates, such as inaccurate roof representations and numerous 'floaters'. UrbanIR addresses these issues with novel losses that reduce errors in inverse graphics inference and rendering artifacts. Its techniques allow for precise shadow volume estimation in the original scene. The model's outputs support controllable editing, enabling photorealistic free-viewpoint renderings of night simulations, relit scenes, and inserted objects,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Remote Sensing and LiDAR Applications
