GaussianBlock: Building Part-Aware Compositional and Editable 3D Scene by Primitives and Gaussians

TL;DR

GaussianBlock introduces a part-aware, compositional 3D scene reconstruction method that combines primitives and 3D Gaussians, enabling interpretable, editable, and high-fidelity scene representations.

Contribution

It proposes a novel hybrid representation with attention-guided primitives and Gaussian refinement, enhancing interpretability and editability of 3D reconstructions.

Findings

Reconstructed scenes are disentangled, compositional, and compact.

Enables precise, seamless editing of 3D scenes.

Achieves high fidelity comparable to state-of-the-art methods.

Abstract

Recently, with the development of Neural Radiance Fields and Gaussian Splatting, 3D reconstruction techniques have achieved remarkably high fidelity. However, the latent representations learnt by these methods are highly entangled and lack interpretability. In this paper, we propose a novel part-aware compositional reconstruction method, called GaussianBlock, that enables semantically coherent and disentangled representations, allowing for precise and physical editing akin to building blocks, while simultaneously maintaining high fidelity. Our GaussianBlock introduces a hybrid representation that leverages the advantages of both primitives, known for their flexible actionability and editability, and 3D Gaussians, which excel in reconstruction quality. Specifically, we achieve semantically coherent primitives through a novel attention-guided centering loss derived from 2D semantic priors, complemented by a dynamic splitting and fusion strategy. Furthermore, we utilize 3D Gaussians that hybridize with primitives to refine structural details and enhance fidelity. Additionally, a binding inheritance strategy is employed to strengthen and maintain the connection between the two. Our reconstructed scenes are evidenced to be disentangled, compositional, and compact across diverse benchmarks, enabling seamless, direct and precise editing while maintaining high quality.