$E^{3}$Gen: Efficient, Expressive and Editable Avatars Generation

TL;DR

E3Gen introduces a novel 3D Gaussian-based avatar generation method that employs a structured UV representation and part-aware deformation to enable efficient, expressive, and editable digital avatars with full-body pose control.

Contribution

The paper proposes a new generative UV features plane for structured 3D Gaussian encoding and a part-aware deformation module for expressive pose control, addressing key challenges in avatar generation.

Findings

Achieves superior avatar generation performance.

Enables expressive full-body pose control.

Supports avatar editing and customization.

Abstract

This paper aims to introduce 3D Gaussian for efficient, expressive, and editable digital avatar generation. This task faces two major challenges: (1) The unstructured nature of 3D Gaussian makes it incompatible with current generation pipelines; (2) the expressive animation of 3D Gaussian in a generative setting that involves training with multiple subjects remains unexplored. In this paper, we propose a novel avatar generation method named $E^3$Gen, to effectively address these challenges. First, we propose a novel generative UV features plane representation that encodes unstructured 3D Gaussian onto a structured 2D UV space defined by the SMPL-X parametric model. This novel representation not only preserves the representation ability of the original 3D Gaussian but also introduces a shared structure among subjects to enable generative learning of the diffusion model. To tackle the second challenge, we propose a part-aware deformation module to achieve robust and accurate full-body expressive pose control. Extensive experiments demonstrate that our method achieves superior performance in avatar generation and enables expressive full-body pose control and editing. Our project page is https://olivia23333.github.io/E3Gen.