TL;DR
Sketch3DVE introduces a novel sketch-based 3D-aware video editing framework that enables detailed local modifications with significant viewpoint changes, ensuring consistency and realism through 3D geometry manipulation and diffusion models.
Contribution
It presents a new method combining sketch-based interaction, 3D geometry estimation, and diffusion models for realistic, viewpoint-consistent video editing from sparse inputs.
Findings
Outperforms existing methods in editing quality and consistency.
Effectively handles large viewpoint changes in videos.
Enables precise local modifications with sketch control.
Abstract
Recent video editing methods achieve attractive results in style transfer or appearance modification. However, editing the structural content of 3D scenes in videos remains challenging, particularly when dealing with significant viewpoint changes, such as large camera rotations or zooms. Key challenges include generating novel view content that remains consistent with the original video, preserving unedited regions, and translating sparse 2D inputs into realistic 3D video outputs. To address these issues, we propose Sketch3DVE, a sketch-based 3D-aware video editing method to enable detailed local manipulation of videos with significant viewpoint changes. To solve the challenge posed by sparse inputs, we employ image editing methods to generate edited results for the first frame, which are then propagated to the remaining frames of the video. We utilize sketching as an interaction tool…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
