GScream: Learning 3D Geometry and Feature Consistent Gaussian Splatting for Object Removal

TL;DR

This paper presents GScream, a novel framework for object removal in 3D scenes that enhances geometric and texture consistency in Gaussian Splatting, improving view synthesis quality and efficiency.

Contribution

The paper introduces a new method that optimizes Gaussian primitive placement and employs feature propagation with cross-attention to improve 3D object removal.

Findings

Improved geometric consistency across removed and visible areas.

Enhanced texture coherence in the radiance field.

Faster training and rendering speeds.

Abstract

This paper tackles the intricate challenge of object removal to update the radiance field using the 3D Gaussian Splatting. The main challenges of this task lie in the preservation of geometric consistency and the maintenance of texture coherence in the presence of the substantial discrete nature of Gaussian primitives. We introduce a robust framework specifically designed to overcome these obstacles. The key insight of our approach is the enhancement of information exchange among visible and invisible areas, facilitating content restoration in terms of both geometry and texture. Our methodology begins with optimizing the positioning of Gaussian primitives to improve geometric consistency across both removed and visible areas, guided by an online registration process informed by monocular depth estimation. Following this, we employ a novel feature propagation mechanism to bolster texture coherence, leveraging a cross-attention design that bridges sampling Gaussians from both uncertain and certain areas. This innovative approach significantly refines the texture coherence within the final radiance field. Extensive experiments validate that our method not only elevates the quality of novel view synthesis for scenes undergoing object removal but also showcases notable efficiency gains in training and rendering speeds.