Controllable Video Object Insertion via Multiview Priors

TL;DR

This paper introduces a novel method for video object insertion that uses multiview priors and view-consistent conditioning to improve appearance, occlusion handling, and temporal coherence.

Contribution

It proposes a new framework that lifts 2D images into multiview representations and employs modules for spatial realism and temporal consistency, addressing key challenges in video object insertion.

Findings

Significantly improves the quality of inserted objects in videos.

Ensures stable identity guidance and robust integration across viewpoints.

Effectively handles occlusion and boundary artifacts while maintaining temporal continuity.

Abstract

Video object insertion is a critical task for dynamically inserting new objects into existing environments. Previous video generation methods focus primarily on synthesizing entire scenes while struggling with ensuring consistent object appearance, spatial alignment, and temporal coherence when inserting objects into existing videos. In this paper, we propose a novel solution for Video Object Insertion, which integrates multi-view object priors to address the common challenges of appearance inconsistency and occlusion handling in dynamic environments. By lifting 2D reference images into multi-view representations and leveraging a dual-path view-consistent conditioning mechanism, our framework ensures stable identity guidance and robust integration across diverse viewpoints. A quality-aware weighting mechanism is also employed to adaptively handle noisy or imperfect inputs. Additionally, we introduce an Integration-Aware Consistency Module that guarantees spatial realism, effectively resolving occlusion and boundary artifacts while maintaining temporal continuity across frames. Experimental results show that our solution significantly improves the quality of video object insertion, providing stable and realistic integration.