3D-GIF: 3D-Controllable Object Generation via Implicit Factorized Representations

TL;DR

This paper introduces a novel 3D object generation method using implicit factorized representations that are view-independent and light-disentangled, enabling realistic, controllable, and relightable 3D object synthesis from 2D images.

Contribution

It proposes a new factorized representation and training scheme that improve 3D geometry quality and enable albedo-textured mesh extraction without extra labels.

Findings

Enhanced 3D geometry quality demonstrated visually and quantitatively.

Ability to re-light images and generate textured meshes from 2D images.

First method to extract textured meshes from unposed 2D images without labels.

Abstract

While NeRF-based 3D-aware image generation methods enable viewpoint control, limitations still remain to be adopted to various 3D applications. Due to their view-dependent and light-entangled volume representation, the 3D geometry presents unrealistic quality and the color should be re-rendered for every desired viewpoint. To broaden the 3D applicability from 3D-aware image generation to 3D-controllable object generation, we propose the factorized representations which are view-independent and light-disentangled, and training schemes with randomly sampled light conditions. We demonstrate the superiority of our method by visualizing factorized representations, re-lighted images, and albedo-textured meshes. In addition, we show that our approach improves the quality of the generated geometry via visualization and quantitative comparison. To the best of our knowledge, this is the first work that extracts albedo-textured meshes with unposed 2D images without any additional labels or assumptions.