ARM: Appearance Reconstruction Model for Relightable 3D Generation

TL;DR

ARM is a novel 3D reconstruction method that separately models geometry and appearance, using UV space processing and material priors to produce more realistic textured 3D meshes from sparse images.

Contribution

It introduces a decoupled geometry-appearance framework with UV space processing and semantic material priors, improving appearance realism in 3D reconstructions.

Findings

ARM outperforms existing methods quantitatively.

ARM produces higher quality, more realistic textures.

Efficient training on 8 GPUs.

Abstract

Recent image-to-3D reconstruction models have greatly advanced geometry generation, but they still struggle to faithfully generate realistic appearance. To address this, we introduce ARM, a novel method that reconstructs high-quality 3D meshes and realistic appearance from sparse-view images. The core of ARM lies in decoupling geometry from appearance, processing appearance within the UV texture space. Unlike previous methods, ARM improves texture quality by explicitly back-projecting measurements onto the texture map and processing them in a UV space module with a global receptive field. To resolve ambiguities between material and illumination in input images, ARM introduces a material prior that encodes semantic appearance information, enhancing the robustness of appearance decomposition. Trained on just 8 H100 GPUs, ARM outperforms existing methods both quantitatively and qualitatively.