HOMER: Homography-Based Efficient Multi-view 3D Object Removal

TL;DR

HOMER introduces a homography-based, efficient multi-view 3D object removal pipeline that improves mask generation and inpainting, achieving state-of-the-art results with significantly reduced runtime across various radiance field frameworks.

Contribution

The paper presents a novel, efficient pipeline for multi-view 3D object removal that does not require camera poses or extra training, and is compatible with multiple radiance field methods.

Findings

Achieves state-of-the-art performance on benchmarks.

Reduces runtime to one-fifth of leading methods.

Demonstrates broad applicability across different radiance field frameworks.

Abstract

3D object removal is an important sub-task in 3D scene editing, with broad applications in scene understanding, augmented reality, and robotics. However, existing methods struggle to achieve a desirable balance among consistency, usability, and computational efficiency in multi-view settings. These limitations are primarily due to unintuitive user interaction in the source view, inefficient multi-view object mask generation, computationally expensive inpainting procedures, and a lack of applicability across different radiance field representations. To address these challenges, we propose a novel pipeline that improves the quality and efficiency of multi-view object mask generation and inpainting. Our method introduces an intuitive region-based interaction mechanism in the source view and eliminates the need for camera poses or extra model training. Our lightweight HoMM module is employed to achieve high-quality multi-view mask propagation with enhanced efficiency. In the inpainting stage, we further reduce computational costs by performing inpainting only on selected key views and propagating the results to other views via homography-based mapping. Our pipeline is compatible with a variety of radiance field frameworks, including NeRF and 3D Gaussian Splatting, demonstrating improved generalizability and practicality in real-world scenarios. Additionally, we present a new 3D multi-object removal dataset with greater object diversity and viewpoint variation than existing datasets. Experiments on public benchmarks and our proposed dataset show that our method achieves state-of-the-art performance while reducing runtime to one-fifth of that required by leading baselines.