VeRi3D: Generative Vertex-based Radiance Fields for 3D Controllable Human Image Synthesis

TL;DR

VeRi3D introduces a vertex-based radiance field model for human image synthesis, enabling high-quality, controllable, and editable 3D human images with improved generalization over previous neural radiance field methods.

Contribution

The paper presents VeRi3D, a novel vertex-based radiance field model that enhances controllability and generalization in 3D human image synthesis using parametric human templates.

Findings

Enables photorealistic human image generation with controllable pose and shape.

Supports part-level editing of generated human images.

Achieves better generalization to novel poses and shapes compared to prior neural radiance fields.

Abstract

Unsupervised learning of 3D-aware generative adversarial networks has lately made much progress. Some recent work demonstrates promising results of learning human generative models using neural articulated radiance fields, yet their generalization ability and controllability lag behind parametric human models, i.e., they do not perform well when generalizing to novel pose/shape and are not part controllable. To solve these problems, we propose VeRi3D, a generative human vertex-based radiance field parameterized by vertices of the parametric human template, SMPL. We map each 3D point to the local coordinate system defined on its neighboring vertices, and use the corresponding vertex feature and local coordinates for mapping it to color and density values. We demonstrate that our simple approach allows for generating photorealistic human images with free control over camera pose, human pose, shape, as well as enabling part-level editing.