Zero-shot High-fidelity and Pose-controllable Character Animation

TL;DR

This paper introduces PoseAnimate, a zero-shot framework for character animation from a single image, achieving high fidelity, temporal coherence, and pose control without extensive training data.

Contribution

The paper presents PoseAnimate, a novel zero-shot I2V method with three modules and a transition algorithm, improving character consistency and detail preservation over existing approaches.

Findings

Outperforms state-of-the-art methods in character consistency

Maintains high temporal coherence in generated videos

Effectively preserves fine details and background fidelity

Abstract

Image-to-video (I2V) generation aims to create a video sequence from a single image, which requires high temporal coherence and visual fidelity. However, existing approaches suffer from inconsistency of character appearances and poor preservation of fine details. Moreover, they require a large amount of video data for training, which can be computationally demanding. To address these limitations, we propose PoseAnimate, a novel zero-shot I2V framework for character animation. PoseAnimate contains three key components: 1) a Pose-Aware Control Module (PACM) that incorporates diverse pose signals into text embeddings, to preserve character-independent content and maintain precise alignment of actions. 2) a Dual Consistency Attention Module (DCAM) that enhances temporal consistency and retains character identity and intricate background details. 3) a Mask-Guided Decoupling Module (MGDM) that refines distinct feature perception abilities, improving animation fidelity by decoupling the character and background. We also propose a Pose Alignment Transition Algorithm (PATA) to ensure smooth action transition. Extensive experiment results demonstrate that our approach outperforms the state-of-the-art training-based methods in terms of character consistency and detail fidelity. Moreover, it maintains a high level of temporal coherence throughout the generated animations.