ReplaceAnything3D:Text-Guided 3D Scene Editing with Compositional Neural Radiance Fields

TL;DR

ReplaceAnything3D (RAM3D) is a novel method for text-guided 3D scene editing that allows for the seamless replacement of objects within a scene while maintaining multi-view consistency and scene integrity.

Contribution

It introduces a new erase-and-replace approach for 3D scene editing using compositional neural radiance fields guided by text prompts, enabling flexible object replacement.

Findings

Effective object replacement in 3D scenes demonstrated

Maintains scene consistency across multiple viewpoints

Versatile across various realistic 3D scenes

Abstract

We introduce ReplaceAnything3D model (RAM3D), a novel text-guided 3D scene editing method that enables the replacement of specific objects within a scene. Given multi-view images of a scene, a text prompt describing the object to replace, and a text prompt describing the new object, our Erase-and-Replace approach can effectively swap objects in the scene with newly generated content while maintaining 3D consistency across multiple viewpoints. We demonstrate the versatility of ReplaceAnything3D by applying it to various realistic 3D scenes, showcasing results of modified foreground objects that are well-integrated with the rest of the scene without affecting its overall integrity.