OMEGAS: Object Mesh Extraction from Large Scenes Guided by Gaussian Segmentation

TL;DR

OMEGAS introduces a novel framework for precise 3D object reconstruction within large scenes, utilizing Gaussian segmentation and generative priors to recover occluded and unseen object parts with high accuracy.

Contribution

The paper presents a new 3D segmentation method based on 2D Gaussian Splatting and a target replenishment technique using generative diffusion priors, advancing object reconstruction in large scenes.

Findings

Outperforms existing methods in accuracy and detail preservation

Effectively reconstructs occluded and unseen object parts

Demonstrates robustness across various complex scenes

Abstract

Recent advancements in 3D reconstruction technologies have paved the way for high-quality and real-time rendering of complex 3D scenes. Despite these achievements, a notable challenge persists: it is difficult to precisely reconstruct specific objects from large scenes. Current scene reconstruction techniques frequently result in the loss of object detail textures and are unable to reconstruct object portions that are occluded or unseen in views. To address this challenge, we delve into the meticulous 3D reconstruction of specific objects within large scenes and propose a framework termed OMEGAS: Object Mesh Extraction from Large Scenes Guided by Gaussian Segmentation. Specifically, we proposed a novel 3D target segmentation technique based on 2D Gaussian Splatting, which segments 3D consistent target masks in multi-view scene images and generates a preliminary target model. Moreover, to reconstruct the unseen portions of the target, we propose a novel target replenishment technique driven by large-scale generative diffusion priors. We demonstrate that our method can accurately reconstruct specific targets from large scenes, both quantitatively and qualitatively. Our experiments show that OMEGAS significantly outperforms existing reconstruction methods across various scenarios. Our project page is at: https://github.com/CrystalWlz/OMEGAS