TL;DR
This paper introduces a novel sketch-guided framework for cartoon video inbetweening that estimates dense cross-domain correspondence and employs blending and occlusion modules to generate high-quality intermediate frames with user-controlled guidance.
Contribution
The approach uniquely combines sketch-guided correspondence estimation with frame interpolation and temporal consistency modules for improved cartoon video synthesis.
Findings
Outperforms existing methods in handling large motions.
Enables user control through sketch editing.
Achieves higher quality and temporal consistency.
Abstract
We propose a novel framework to produce cartoon videos by fetching the color information from two input keyframes while following the animated motion guided by a user sketch. The key idea of the proposed approach is to estimate the dense cross-domain correspondence between the sketch and cartoon video frames, and employ a blending module with occlusion estimation to synthesize the middle frame guided by the sketch. After that, the input frames and the synthetic frame equipped with established correspondence are fed into an arbitrary-time frame interpolation pipeline to generate and refine additional inbetween frames. Finally, a module to preserve temporal consistency is employed. Compared to common frame interpolation methods, our approach can address frames with relatively large motion and also has the flexibility to enable users to control the generated video sequences by editing the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
