Generative Inbetweening through Frame-wise Conditions-Driven Video Generation
Tianyi Zhu, Dongwei Ren, Qilong Wang, Xiaohe Wu, Wangmeng Zuo

TL;DR
This paper introduces FCVG, a simple yet effective method for generative inbetweening that improves temporal stability in video frame interpolation by providing explicit frame-wise conditions, leading to more coherent and plausible videos.
Contribution
The paper proposes a novel Frame-wise Conditions-driven Video Generation (FCVG) approach that explicitly conditions each frame, enhancing temporal stability in video interpolation tasks.
Findings
FCVG significantly improves temporal stability across diverse scenarios.
The method works with both linear and non-linear interpolation curves.
Extensive evaluations show superior coherence compared to existing methods.
Abstract
Generative inbetweening aims to generate intermediate frame sequences by utilizing two key frames as input. Although remarkable progress has been made in video generation models, generative inbetweening still faces challenges in maintaining temporal stability due to the ambiguous interpolation path between two key frames. This issue becomes particularly severe when there is a large motion gap between input frames. In this paper, we propose a straightforward yet highly effective Frame-wise Conditions-driven Video Generation (FCVG) method that significantly enhances the temporal stability of interpolated video frames. Specifically, our FCVG provides an explicit condition for each frame, making it much easier to identify the interpolation path between two input frames and thus ensuring temporally stable production of visually plausible video frames. To achieve this, we suggest extracting…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Generative Adversarial Networks and Image Synthesis · Image Enhancement Techniques
