AnimeCeleb: Large-Scale Animation CelebHeads Dataset for Head Reenactment

TL;DR

This paper introduces AnimeCeleb, a large-scale dataset for animation head reenactment using 3D models, enabling high-quality results and a novel cross-domain pose transfer method for animation heads.

Contribution

We created AnimeCeleb dataset with 3D model-based annotations and propose a new pose mapping architecture for cross-domain head reenactment.

Findings

AnimeCeleb enables high-quality animation head reenactment.

Our cross-domain model outperforms existing methods.

The dataset and code facilitate further research in animation head reenactment.

Abstract

We present a novel Animation CelebHeads dataset (AnimeCeleb) to address an animation head reenactment. Different from previous animation head datasets, we utilize 3D animation models as the controllable image samplers, which can provide a large amount of head images with their corresponding detailed pose annotations. To facilitate a data creation process, we build a semi-automatic pipeline leveraging an open 3D computer graphics software with a developed annotation system. After training with the AnimeCeleb, recent head reenactment models produce high-quality animation head reenactment results, which are not achievable with existing datasets. Furthermore, motivated by metaverse application, we propose a novel pose mapping method and architecture to tackle a cross-domain head reenactment task. During inference, a user can easily transfer one's motion to an arbitrary animation head. Experiments demonstrate the usefulness of the AnimeCeleb to train animation head reenactment models, and the superiority of our cross-domain head reenactment model compared to state-of-the-art methods. Our dataset and code are available at https://github.com/kangyeolk/AnimeCeleb.