MetaHead: An Engine to Create Realistic Digital Head

TL;DR

MetaHead is a comprehensive digital head engine that produces highly realistic, controllable 3D head models and labeled images, improving data generation for face analysis tasks beyond existing methods.

Contribution

It introduces MetaHead, a unified framework combining a controllable radiance field and a label-based image generator for realistic, view-consistent digital heads.

Findings

Achieves state-of-the-art visual quality and reconstruction accuracy.

Generated data surpasses graphics-based methods in training effectiveness.

Enables realistic, controllable 3D head generation for face analysis.

Abstract

Collecting and labeling training data is one important step for learning-based methods because the process is time-consuming and biased. For face analysis tasks, although some generative models can be used to generate face data, they can only achieve a subset of generation diversity, reconstruction accuracy, 3D consistency, high-fidelity visual quality, and easy editability. One recent related work is the graphics-based generative method, but it can only render low realism head with high computation cost. In this paper, we propose MetaHead, a unified and full-featured controllable digital head engine, which consists of a controllable head radiance field(MetaHead-F) to super-realistically generate or reconstruct view-consistent 3D controllable digital heads and a generic top-down image generation framework LabelHead to generate digital heads consistent with the given customizable feature labels. Experiments validate that our controllable digital head engine achieves the state-of-the-art generation visual quality and reconstruction accuracy. Moreover, the generated labeled data can assist real training data and significantly surpass the labeled data generated by graphics-based methods in terms of training effect.