Motion Avatar: Generate Human and Animal Avatars with Arbitrary Motion

TL;DR

This paper introduces Motion Avatar, a comprehensive system for generating customizable 3D human and animal avatars with motions from text, integrating avatar and motion creation, and providing a large animal motion dataset to advance the field.

Contribution

The paper presents a novel agent-based approach for avatar and motion generation, a language model planner for coordination, and a large animal motion dataset, addressing key challenges in 3D avatar creation.

Findings

Achieved high-quality avatar and motion generation from text queries.

Created Zoo-300K, a large-scale animal motion dataset with 300,000 pairs.

Demonstrated the effectiveness of the approach in dynamic 3D character creation.

Abstract

In recent years, there has been significant interest in creating 3D avatars and motions, driven by their diverse applications in areas like film-making, video games, AR/VR, and human-robot interaction. However, current efforts primarily concentrate on either generating the 3D avatar mesh alone or producing motion sequences, with integrating these two aspects proving to be a persistent challenge. Additionally, while avatar and motion generation predominantly target humans, extending these techniques to animals remains a significant challenge due to inadequate training data and methods. To bridge these gaps, our paper presents three key contributions. Firstly, we proposed a novel agent-based approach named Motion Avatar, which allows for the automatic generation of high-quality customizable human and animal avatars with motions through text queries. The method significantly advanced the progress in dynamic 3D character generation. Secondly, we introduced a LLM planner that coordinates both motion and avatar generation, which transforms a discriminative planning into a customizable Q&A fashion. Lastly, we presented an animal motion dataset named Zoo-300K, comprising approximately 300,000 text-motion pairs across 65 animal categories and its building pipeline ZooGen, which serves as a valuable resource for the community. See project website https://steve-zeyu-zhang.github.io/MotionAvatar/