HRGS: Hierarchical Gaussian Splatting for Memory-Efficient High-Resolution 3D Reconstruction
Changbai Li, Haodong Zhu, Hanlin Chen, Juan Zhang, Tongfei Chen, Shuo Yang, Shuwei Shao, Wenhao Dong, Baochang Zhang
TL;DR
HRGS introduces a hierarchical, memory-efficient approach for high-resolution 3D scene reconstruction, combining coarse-to-fine Gaussian representations with importance-driven pruning and normal priors to achieve state-of-the-art results.
Contribution
The paper presents Hierarchical Gaussian Splatting (HRGS), a novel framework that improves memory efficiency and reconstruction quality in high-resolution 3D scene reconstruction.
Findings
Achieves state-of-the-art high-resolution NVS performance
Reduces memory usage significantly compared to existing methods
Enables high-quality surface reconstruction under memory constraints
Abstract
3D Gaussian Splatting (3DGS) has made significant strides in real-time 3D scene reconstruction, but faces memory scalability issues in high-resolution scenarios. To address this, we propose Hierarchical Gaussian Splatting (HRGS), a memory-efficient framework with hierarchical block-level optimization. First, we generate a global, coarse Gaussian representation from low-resolution data. Then, we partition the scene into multiple blocks, refining each block with high-resolution data. The partitioning involves two steps: Gaussian partitioning, where irregular scenes are normalized into a bounded cubic space with a uniform grid for task distribution, and training data partitioning, where only relevant observations are retained for each block. By guiding block refinement with the coarse Gaussian prior, we ensure seamless Gaussian fusion across adjacent blocks. To reduce computational…
Peer Reviews
Decision·ICLR 2026 Conference Withdrawn Submission
HRGS is much faster and more memory-efficient compared to competing methods for novel-view synthesis and surface reconstruction (Table 1-3, Figure C, Table F). At the same time, it achieves state-of-the-art image quality and F1 scores on the Mip-NeRF 360 and Replica datasets, respectively. Being able to do high-resolution novel-view synthesis in a memory-efficient way is a very impactful research area, in my opinion. Consumer GPUs (e.g. with 24 GB of VRAM) frequently do not have enough memory
I am not fully convinced that the contraction algorithm will work well for scenes that are larger or sparser in camera views (e.g. a long, curved corridor). The contraction algorithm assumes an "internal region" of the scene bounding box to be at the center (L245), but the center of the scene may be empty for some scenes (e.g. a circular path around a building external). Additionally, I belief the nonlinear mapping rule only applies a scalar scaling (Eq. 2) of points from the origin, which again
- Directly and effectively addresses the major scalability issue of 3DGS, enabling high-resolution (5K) reconstruction on commodity, memory-constrained GPUs where other methods fail. - The paper proposes an effective contrastive method to ensure that the Gaussians are properly separated, providing a more reasonable and principled paradigm. - The performance gains are clear and substantial. HRGS achieves superior PSNR and SSIM on the Mip-NeRF360 dataset while using nearly half the GPU memory of
- The two-stage, block-wise process is inherently more complex and slower than a single global optimization. The paper reports a training time of 5 hours for HRGS vs. 3 hours for 3DGS, representing a trade-off of time for memory efficiency. I also suggest reporting comparisons using the same GPU type and normalized GPU hours to ensure fairness in evaluating efficiency. - The proposed Importance-Driven Gaussian Pruning (IDGP) induces resolution-dependent information loss, disproportionately remov
The proposed concept is sound, and addresses a critical issue of scaling 3DGS when it's nontrivial to breakup the gaussians properly. The overall approach shows some metric improvement in NVS and surface reconstruction. The authors show that this is done with a balance of producing high resolution detail and low memory footprint on standard benchmark.
I think this work, while has good motivation, suffers from trying to achieve multiple things at the same time, with confusing message in the end. It is unclear to me if the authors want to focus on e.g., efficient GS that leads to HR rendering, or surface reconstruction. I would be interested to see, e.g., just by scaling GS subdivision, how far can we get to the best performance possible. In my mind, if I have infinite memory, I should be able to scale GS to densify as much as it needs to, and
1. Addresses a relevant problem: scaling 3D Gaussian Splatting to high-resolution scenes under limited GPU memory. 2. Integrates known concepts (hierarchical refinement, pruning, data partitioning) into a unified training pipeline. 3. Provides extensive comparisons across multiple datasets with standard metrics.
1. Misaligned comparison with Mip-Splatting: The paper repeatedly compares HRGS against Mip-Splatting (Yu et al., 2024) and even highlights marginal improvements in PSNR and SSIM. However, this comparison is conceptually inappropriate: Mip-Splatting is primarily designed for alias-free radiance rendering, not for memory-efficient or block-wise optimization. 2. Incomplete ablation and missing baselines: Although the appendix mentions additional ablation studies, the main text provides only brief
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMedical Image Segmentation Techniques · Computer Graphics and Visualization Techniques · 3D Shape Modeling and Analysis
MethodsPruning