TL;DR
Animate-X++ is a versatile framework that animates various character types with dynamic backgrounds, leveraging advanced motion modeling and multi-task training to produce realistic videos, including anthropomorphic characters, outperforming previous methods.
Contribution
The paper introduces Animate-X++, a novel universal animation framework that generalizes to different character types and incorporates dynamic backgrounds, with new motion modeling and training strategies.
Findings
Outperforms existing methods in animation quality and realism.
Successfully animates anthropomorphic characters and dynamic backgrounds.
Introduces A2Bench for comprehensive evaluation.
Abstract
Character image animation, which generates high-quality videos from a reference image and target pose sequence, has seen significant progress in recent years. However, most existing methods only apply to human figures, which usually do not generalize well on anthropomorphic characters commonly used in industries like gaming and entertainment. Furthermore, previous methods could only generate videos with static backgrounds, which limits the realism of the videos. For the first challenge, our in-depth analysis suggests to attribute this limitation to their insufficient modeling of motion, which is unable to comprehend the movement pattern of the driving video, thus imposing a pose sequence rigidly onto the target character. To this end, this paper proposes Animate-X++, a universal animation framework based on DiT for various character types, including anthropomorphic characters. To…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
