MVAnimate: Enhancing Character Animation with Multi-View Optimization

TL;DR

MVAnimate is a new framework that combines multi-view 2D and 3D data to improve the quality, consistency, and realism of character animations, addressing issues of low quality and data scarcity.

Contribution

It introduces a novel multi-view optimization approach that enhances animation quality by leveraging multi-view prior information for both 2D and 3D data.

Findings

Improves animation quality over existing methods

Ensures temporal consistency and spatial coherence

Robust across diverse motion patterns

Abstract

The demand for realistic and versatile character animation has surged, driven by its wide-ranging applications in various domains. However, the animation generation algorithms modeling human pose with 2D or 3D structures all face various problems, including low-quality output content and training data deficiency, preventing the related algorithms from generating high-quality animation videos. Therefore, we introduce MVAnimate, a novel framework that synthesizes both 2D and 3D information of dynamic figures based on multi-view prior information, to enhance the generated video quality. Our approach leverages multi-view prior information to produce temporally consistent and spatially coherent animation outputs, demonstrating improvements over existing animation methods. Our MVAnimate also optimizes the multi-view videos of the target character, enhancing the video quality from different views. Experimental results on diverse datasets highlight the robustness of our method in handling various motion patterns and appearances.