Flow-Guided Video Inpainting with Scene Templates
Dong Lao, Peihao Zhu, Peter Wonka, Ganesh Sundaramoorthi

TL;DR
This paper introduces a flow-guided video inpainting method that uses scene templates and a novel interpolation scheme to produce more consistent and less distorted inpainted videos, outperforming existing methods.
Contribution
The paper presents a novel flow-based video inpainting approach that jointly infers scene templates and mappings, improving temporal consistency and reducing artifacts.
Findings
Outperforms state-of-the-art methods on benchmark datasets
Produces crisper inpaintings with fewer distortions
Validated through quantitative metrics and user studies
Abstract
We consider the problem of filling in missing spatio-temporal regions of a video. We provide a novel flow-based solution by introducing a generative model of images in relation to the scene (without missing regions) and mappings from the scene to images. We use the model to jointly infer the scene template, a 2D representation of the scene, and the mappings. This ensures consistency of the frame-to-frame flows generated to the underlying scene, reducing geometric distortions in flow based inpainting. The template is mapped to the missing regions in the video by a new L2-L1 interpolation scheme, creating crisp inpaintings and reducing common blur and distortion artifacts. We show on two benchmark datasets that our approach out-performs state-of-the-art quantitatively and in user studies.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
