GOR-IS: 3D Gaussian Object Removal in the Intrinsic Space

TL;DR

The paper introduces GOR-IS, a novel 3D object removal framework that ensures physical consistency and visual coherence by modeling intrinsic scene components and global lighting effects, outperforming existing methods.

Contribution

It presents a new intrinsic-space inpainting approach that explicitly models light transport and handles non-Lambertian surfaces for 3D object removal.

Findings

Improves perceptual similarity (LPIPS) by 13% over previous methods.

Enhances PSNR by 2dB compared to existing approaches.

Demonstrates effectiveness on synthetic and real-world datasets.

Abstract

Recent advances in Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have made it standard practice to reconstruct 3D scenes from multi-view images. Removing objects from such 3D representations is a fundamental editing task that requires complete and seamless inpainting of occluded regions, ensuring consistency in geometry and appearance. Although existing methods have made notable progress in improving inpainting consistency, they often neglect global lighting effects, leading to physically implausible results. Moreover, these methods struggle with view-dependent non-Lambertian surfaces, where appearance varies across viewpoints, leading to unreliable inpainting. In this paper, we present 3D Gaussian Object Removal in the Intrinsic Space (GOR-IS), a novel framework for physically consistent and visually coherent 3D object removal. Our approach decomposes the scene into intrinsic components and explicitly models light transport to maintain global lighting effects consistency. Furthermore, we introduce an intrinsic-space inpainting module that operates directly in the material and lighting domains, effectively addressing the challenges posed by non-Lambertian surfaces. Extensive experiments on both synthetic and real-world datasets demonstrate that our framework substantially improves the physical consistency and visual coherence of object removal, outperforming existing methods by 13% in perceptual similarity (LPIPS) and 2dB in peak signal-to-noise ratio (PSNR). Code is publicly available at https://applezyh.github.io/GOR-IS-project-page/