Make-A-Character 2: Animatable 3D Character Generation From a Single Image

TL;DR

Make-A-Character 2 is a fast, neural network-based system that creates high-quality, animatable 3D characters from a single image, suitable for interactive applications and digital humans.

Contribution

It introduces novel image correction, hierarchical facial modeling, and real-time gesture generation techniques for single-image 3D character creation.

Findings

Generation process takes less than 2 minutes.

Produces high-quality, animatable 3D characters from single images.

Enables real-time conversational interactions with generated characters.

Abstract

This report introduces Make-A-Character 2, an advanced system for generating high-quality 3D characters from single portrait photographs, ideal for game development and digital human applications. Make-A-Character 2 builds upon its predecessor by incorporating several significant improvements for image-based head generation. We utilize the IC-Light method to correct non-ideal illumination in input photos and apply neural network-based color correction to harmonize skin tones between the photos and game engine renders. We also employ the Hierarchical Representation Network to capture high-frequency facial structures and conduct adaptive skeleton calibration for accurate and expressive facial animations. The entire image-to-3D-character generation process takes less than 2 minutes. Furthermore, we leverage transformer architecture to generate co-speech facial and gesture actions, enabling real-time conversation with the generated character. These technologies have been integrated into our conversational AI avatar products.