VEIGAR: View-consistent Explicit Inpainting and Geometry Alignment for 3D object Removal

TL;DR

VEIGAR is a novel, efficient framework for 3D object removal that ensures view consistency and high-quality reconstruction without initial 3D reconstruction, using a lightweight model and scale-invariant depth supervision.

Contribution

It introduces VEIGAR, a new method that eliminates the need for initial 3D reconstruction and employs a scale-invariant depth loss for improved efficiency and quality.

Findings

Sets new state-of-the-art in reconstruction quality and cross-view consistency.

Reduces training time by threefold compared to existing methods.

Achieves superior efficiency and effectiveness in 3D object removal.

Abstract

Recent advances in Novel View Synthesis (NVS) and 3D generation have significantly improved editing tasks, with a primary emphasis on maintaining cross-view consistency throughout the generative process. Contemporary methods typically address this challenge using a dual-strategy framework: performing consistent 2D inpainting across all views guided by embedded priors either explicitly in pixel space or implicitly in latent space; and conducting 3D reconstruction with additional consistency guidance. Previous strategies, in particular, often require an initial 3D reconstruction phase to establish geometric structure, introducing considerable computational overhead. Even with the added cost, the resulting reconstruction quality often remains suboptimal. In this paper, we present VEIGAR, a computationally efficient framework that outperforms existing methods without relying on an initial reconstruction phase. VEIGAR leverages a lightweight foundation model to reliably align priors explicitly in the pixel space. In addition, we introduce a novel supervision strategy based on scale-invariant depth loss, which removes the need for traditional scale-and-shift operations in monocular depth regularization. Through extensive experimentation, VEIGAR establishes a new state-of-the-art benchmark in reconstruction quality and cross-view consistency, while achieving a threefold reduction in training time compared to the fastest existing method, highlighting its superior balance of efficiency and effectiveness.