Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance

TL;DR

Arc2Avatar is a novel method that generates expressive, realistic 3D human avatars from a single image using foundation models and a specialized 3D Gaussian Splatting approach, enabling detailed and diverse expressions.

Contribution

It introduces the first SDS-based approach for 3D avatar generation from a single image guided by a human face foundation model, with improved realism and expression control.

Findings

Achieves state-of-the-art realism and identity preservation.

Effectively generates diverse expressions with low guidance.

Refines expressions using an optional SDS-based correction step.

Abstract

Inspired by the effectiveness of 3D Gaussian Splatting (3DGS) in reconstructing detailed 3D scenes within multi-view setups and the emergence of large 2D human foundation models, we introduce Arc2Avatar, the first SDS-based method utilizing a human face foundation model as guidance with just a single image as input. To achieve that, we extend such a model for diverse-view human head generation by fine-tuning on synthetic data and modifying its conditioning. Our avatars maintain a dense correspondence with a human face mesh template, allowing blendshape-based expression generation. This is achieved through a modified 3DGS approach, connectivity regularizers, and a strategic initialization tailored for our task. Additionally, we propose an optional efficient SDS-based correction step to refine the blendshape expressions, enhancing realism and diversity. Experiments demonstrate that Arc2Avatar achieves state-of-the-art realism and identity preservation, effectively addressing color issues by allowing the use of very low guidance, enabled by our strong identity prior and initialization strategy, without compromising detail. Please visit https://arc2avatar.github.io for more resources.