Enhancing Video Inpainting with Aligned Frame Interval Guidance
Ming Xie, Junqiu Yu, Qiaole Dong, Xiangyang Xue, Yanwei Fu

TL;DR
VidPivot introduces a novel video inpainting framework that utilizes aligned frame interval guidance and a content propagation module to improve spatiotemporal coherence and reduce content degradation.
Contribution
The paper proposes VidPivot, which decouples video inpainting into image inpainting and motion propagation, using frame interval priors and a dedicated context controller for enhanced coherence.
Findings
Achieves competitive performance on multiple benchmarks.
Effectively reduces content degradation within video chunks.
Generalizes well across various video inpainting scenarios.
Abstract
Recent image-to-video (I2V) based video inpainting methods have made significant strides by leveraging single-image priors and modeling temporal consistency across masked frames. Nevertheless, these methods suffer from severe content degradation within video chunks. Furthermore, the absence of a robust frame alignment scheme compromises intra-chunk and inter-chunk spatiotemporal stability, resulting in insufficient control over the entire video. To address these limitations, we propose VidPivot, a novel framework that decouples video inpainting into two sub-tasks: multi-frame consistent image inpainting and masked area motion propagation. Our approach introduces frame interval priors as spatiotemporal cues to guide the inpainting process. To enhance cross-frame coherence, we design a FrameProp Module that implements a frame content propagation strategy, diffusing reference frame content…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGenerative Adversarial Networks and Image Synthesis · Advanced Image Processing Techniques · Image Enhancement Techniques
