Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects

TL;DR

Instant3dit introduces a rapid 3D shape editing method that leverages multiview image inpainting, enabling edits in approximately 3 seconds with higher quality than previous approaches.

Contribution

The paper presents a novel multiview inpainting approach for 3D editing that significantly reduces editing time and improves quality without extensive optimization.

Findings

Editing time reduced from hours to seconds

Achieves higher-quality 3D edits compared to prior methods

Effective multiview inpainting strategies for 3D shape editing

Abstract

We propose a generative technique to edit 3D shapes, represented as meshes, NeRFs, or Gaussian Splats, in approximately 3 seconds, without the need for running an SDS type of optimization. Our key insight is to cast 3D editing as a multiview image inpainting problem, as this representation is generic and can be mapped back to any 3D representation using the bank of available Large Reconstruction Models. We explore different fine-tuning strategies to obtain both multiview generation and inpainting capabilities within the same diffusion model. In particular, the design of the inpainting mask is an important factor of training an inpainting model, and we propose several masking strategies to mimic the types of edits a user would perform on a 3D shape. Our approach takes 3D generative editing from hours to seconds and produces higher-quality results compared to previous works.