Nuvo: Neural UV Mapping for Unruly 3D Representations

TL;DR

Nuvo introduces a neural field-based UV mapping technique tailored for complex 3D reconstructions, enabling continuous, valid, and editable UV maps directly from challenging geometry representations.

Contribution

It proposes a novel neural UV mapping method that operates on neural radiance fields and similar geometries, overcoming limitations of traditional mesh-based approaches.

Findings

Produces continuous UV maps from complex 3D data

Robust to ill-behaved geometry

Enables detailed and editable textures

Abstract

Existing UV mapping algorithms are designed to operate on well-behaved meshes, instead of the geometry representations produced by state-of-the-art 3D reconstruction and generation techniques. As such, applying these methods to the volume densities recovered by neural radiance fields and related techniques (or meshes triangulated from such fields) results in texture atlases that are too fragmented to be useful for tasks such as view synthesis or appearance editing. We present a UV mapping method designed to operate on geometry produced by 3D reconstruction and generation techniques. Instead of computing a mapping defined on a mesh's vertices, our method Nuvo uses a neural field to represent a continuous UV mapping, and optimizes it to be a valid and well-behaved mapping for just the set of visible points, i.e. only points that affect the scene's appearance. We show that our model is robust to the challenges posed by ill-behaved geometry, and that it produces editable UV mappings that can represent detailed appearance.