LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditions

TL;DR

LightCity is a comprehensive synthetic urban dataset designed to facilitate research on inverse rendering and 3D reconstruction under complex multi-illumination conditions, addressing a key gap in existing datasets.

Contribution

The paper introduces LightCity, a high-quality, controllable synthetic dataset with diverse illumination, supporting benchmarking and analysis of inverse rendering in urban scenes.

Findings

Benchmarking three fundamental urban tasks.

Analysis of illumination effects on reconstruction.

Dataset enables new research directions.

Abstract

Inverse rendering in urban scenes is pivotal for applications like autonomous driving and digital twins. Yet, it faces significant challenges due to complex illumination conditions, including multi-illumination and indirect light and shadow effects. However, the effects of these challenges on intrinsic decomposition and 3D reconstruction have not been explored due to the lack of appropriate datasets. In this paper, we present LightCity, a novel high-quality synthetic urban dataset featuring diverse illumination conditions with realistic indirect light and shadow effects. LightCity encompasses over 300 sky maps with highly controllable illumination, varying scales with street-level and aerial perspectives over 50K images, and rich properties such as depth, normal, material components, light and indirect light, etc. Besides, we leverage LightCity to benchmark three fundamental tasks in the urban environments and conduct a comprehensive analysis of these benchmarks, laying a robust foundation for advancing related research.