Sketch Me A Video
Haichao Zhang, Gang Yu, Tao Chen, Guozhong Luo

TL;DR
This paper introduces a novel two-stage Sketch-to-Video framework that synthesizes realistic portrait videos from just two rough sketches, overcoming the lack of guiding videos and handling free-form sketch styles.
Contribution
The work proposes a feature retrieve and projection module and a motion projection with feature blending, addressing sketch style variability and missing temporal guidance in video synthesis.
Findings
Achieves high-quality video synthesis from rough sketches.
Outperforms existing methods in both quantitative and qualitative evaluations.
Effectively handles free-form sketch styles and missing temporal information.
Abstract
Video creation has been an attractive yet challenging task for artists to explore. With the advancement of deep learning, recent works try to utilize deep convolutional neural networks to synthesize a video with the aid of a guiding video, and have achieved promising results. However, the acquisition of guiding videos, or other forms of guiding temporal information is costly expensive and difficult in reality. Therefore, in this work we introduce a new video synthesis task by employing two rough bad-drwan sketches only as input to create a realistic portrait video. A two-stage Sketch-to-Video model is proposed, which consists of two key novelties: 1) a feature retrieve and projection (FRP) module, which parititions the input sketch into different parts and utilizes these parts for synthesizing a realistic start or end frame and meanwhile generating rich semantic features, is designed to…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Computer Graphics and Visualization Techniques · Human Motion and Animation
MethodsTest
