FlexiClip: Locality-Preserving Free-Form Character Animation

TL;DR

FlexiClip introduces a novel method for clipart animation that ensures smooth, natural, and structurally consistent motion by integrating advanced temporal and geometric modeling techniques with pre-trained diffusion models.

Contribution

The paper presents FlexiClip, a new approach that combines Bézier trajectories, temporal Jacobians, pfODEs, and flow matching loss to improve clipart animation quality and consistency.

Findings

FlexiClip produces smoother and more natural animations.

It maintains structural consistency across diverse clipart types.

The method outperforms existing techniques in visual fidelity and temporal coherence.

Abstract

Animating clipart images with seamless motion while maintaining visual fidelity and temporal coherence presents significant challenges. Existing methods, such as AniClipart, effectively model spatial deformations but often fail to ensure smooth temporal transitions, resulting in artifacts like abrupt motions and geometric distortions. Similarly, text-to-video (T2V) and image-to-video (I2V) models struggle to handle clipart due to the mismatch in statistical properties between natural video and clipart styles. This paper introduces FlexiClip, a novel approach designed to overcome these limitations by addressing the intertwined challenges of temporal consistency and geometric integrity. FlexiClip extends traditional B\'ezier curve-based trajectory modeling with key innovations: temporal Jacobians to correct motion dynamics incrementally, continuous-time modeling via probability flow ODEs (pfODEs) to mitigate temporal noise, and a flow matching loss inspired by GFlowNet principles to optimize smooth motion transitions. These enhancements ensure coherent animations across complex scenarios involving rapid movements and non-rigid deformations. Extensive experiments validate the effectiveness of FlexiClip in generating animations that are not only smooth and natural but also structurally consistent across diverse clipart types, including humans and animals. By integrating spatial and temporal modeling with pre-trained video diffusion models, FlexiClip sets a new standard for high-quality clipart animation, offering robust performance across a wide range of visual content. Project Page: https://creative-gen.github.io/flexiclip.github.io/