AttentionGS: Towards Initialization-Free 3D Gaussian Splatting via Structural Attention

TL;DR

AttentionGS introduces a novel method for 3D Gaussian Splatting that removes the need for high-quality initial point clouds by using structural attention, improving robustness and reconstruction quality in challenging scenarios.

Contribution

It proposes a structural attention framework that enables direct 3D reconstruction from random initialization, eliminating reliance on SfM point clouds.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Effective in scenarios with unreliable point cloud initialization.

Enhances 3D reconstruction quality and rendering robustness.

Abstract

3D Gaussian Splatting (3DGS) is a powerful alternative to Neural Radiance Fields (NeRF), excelling in complex scene reconstruction and efficient rendering. However, it relies on high-quality point clouds from Structure-from-Motion (SfM), limiting its applicability. SfM also fails in texture-deficient or constrained-view scenarios, causing severe degradation in 3DGS reconstruction. To address this limitation, we propose AttentionGS, a novel framework that eliminates the dependency on high-quality initial point clouds by leveraging structural attention for direct 3D reconstruction from randomly initialization. In the early training stage, we introduce geometric attention to rapidly recover the global scene structure. As training progresses, we incorporate texture attention to refine fine-grained details and enhance rendering quality. Furthermore, we employ opacity-weighted gradients to guide Gaussian densification, leading to improved surface reconstruction. Extensive experiments on multiple benchmark datasets demonstrate that AttentionGS significantly outperforms state-of-the-art methods, particularly in scenarios where point cloud initialization is unreliable. Our approach paves the way for more robust and flexible 3D Gaussian Splatting in real-world applications.