Projective Urban Texturing

TL;DR

This paper introduces Projective Urban Texturing (PUT), a neural method for automatically generating realistic textures for 3D city meshes by transferring style from panoramic urban images, addressing challenges of shape mismatch and object variability.

Contribution

We develop PUT, a novel neural framework that re-targets urban textural styles from panoramic images to 3D meshes using unpaired image-to-texture translation with iterative consistency.

Findings

Textures are visually realistic and style-consistent.

Quantitative metrics show improved style transfer accuracy.

Qualitative results demonstrate applicability to diverse urban scenes.

Abstract

This paper proposes a method for automatic generation of textures for 3D city meshes in immersive urban environments. Many recent pipelines capture or synthesize large quantities of city geometry using scanners or procedural modeling pipelines. Such geometry is intricate and realistic, however the generation of photo-realistic textures for such large scenes remains a problem. We propose to generate textures for input target 3D meshes driven by the textural style present in readily available datasets of panoramic photos capturing urban environments. Re-targeting such 2D datasets to 3D geometry is challenging because the underlying shape, size, and layout of the urban structures in the photos do not correspond to the ones in the target meshes. Photos also often have objects (e.g., trees, vehicles) that may not even be present in the target geometry. To address these issues we present a method, called Projective Urban Texturing (PUT), which re-targets textural style from real-world panoramic images to unseen urban meshes. PUT relies on contrastive and adversarial training of a neural architecture designed for unpaired image-to-texture translation. The generated textures are stored in a texture atlas applied to the target 3D mesh geometry. To promote texture consistency, PUT employs an iterative procedure in which texture synthesis is conditioned on previously generated, adjacent textures. We demonstrate both quantitative and qualitative evaluation of the generated textures.