FlipSketch: Flipping Static Drawings to Text-Guided Sketch Animations
Hmrishav Bandyopadhyay, Yi-Zhe Song

TL;DR
FlipSketch is a system that enables users to create sketch animations easily by drawing and describing motion, leveraging text-to-video diffusion models with novel adaptations for sketch style and visual consistency.
Contribution
The paper introduces FlipSketch, a novel method that adapts text-to-video diffusion models for sketch animation through fine-tuning, reference frames, and dual-attention composition.
Findings
Supports dynamic sketch transformations with raster frames
Achieves fluid motion while preserving visual consistency
Simplifies sketch animation to doodling and describing
Abstract
Sketch animations offer a powerful medium for visual storytelling, from simple flip-book doodles to professional studio productions. While traditional animation requires teams of skilled artists to draw key frames and in-between frames, existing automation attempts still demand significant artistic effort through precise motion paths or keyframe specification. We present FlipSketch, a system that brings back the magic of flip-book animation -- just draw your idea and describe how you want it to move! Our approach harnesses motion priors from text-to-video diffusion models, adapting them to generate sketch animations through three key innovations: (i) fine-tuning for sketch-style frame generation, (ii) a reference frame mechanism that preserves visual integrity of input sketch through noise refinement, and (iii) a dual-attention composition that enables fluid motion without losing visual…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Motion and Animation · Augmented Reality Applications · Educational Games and Gamification
MethodsDiffusion
