EVA3D: Compositional 3D Human Generation from 2D Image Collections

TL;DR

EVA3D is a novel 3D human generative model that learns from 2D images, producing detailed, high-quality 3D human models with a compositional NeRF approach, enabling efficient training and realistic rendering.

Contribution

The paper introduces EVA3D, a compositional 3D human generation framework from 2D images, featuring a local-part NeRF representation and pose-guided sampling for improved quality and scalability.

Findings

Achieves state-of-the-art geometry and texture quality in 3D human generation.

Effectively models diverse human poses and appearances.

Demonstrates scalable and realistic 3D human synthesis from 2D collections.

Abstract

Inverse graphics aims to recover 3D models from 2D observations. Utilizing differentiable rendering, recent 3D-aware generative models have shown impressive results of rigid object generation using 2D images. However, it remains challenging to generate articulated objects, like human bodies, due to their complexity and diversity in poses and appearances. In this work, we propose, EVA3D, an unconditional 3D human generative model learned from 2D image collections only. EVA3D can sample 3D humans with detailed geometry and render high-quality images (up to 512x256) without bells and whistles (e.g. super resolution). At the core of EVA3D is a compositional human NeRF representation, which divides the human body into local parts. Each part is represented by an individual volume. This compositional representation enables 1) inherent human priors, 2) adaptive allocation of network parameters, 3) efficient training and rendering. Moreover, to accommodate for the characteristics of sparse 2D human image collections (e.g. imbalanced pose distribution), we propose a pose-guided sampling strategy for better GAN learning. Extensive experiments validate that EVA3D achieves state-of-the-art 3D human generation performance regarding both geometry and texture quality. Notably, EVA3D demonstrates great potential and scalability to "inverse-graphics" diverse human bodies with a clean framework.