Multimodal-Prior-Guided Importance Sampling for Hierarchical Gaussian Splatting in Sparse-View Novel View Synthesis

TL;DR

This paper introduces a multimodal-prior-guided importance sampling method for hierarchical Gaussian Splatting, improving sparse-view novel view synthesis by effectively combining cues and focusing refinement on critical regions.

Contribution

It proposes a novel sampling mechanism that fuses multimodal cues to guide hierarchical Gaussian Splatting, enhancing reconstruction quality in sparse-view scenarios.

Findings

Achieves state-of-the-art reconstruction quality on diverse benchmarks.

Improves PSNR by up to +0.3 dB on DTU dataset.

Effectively suppresses noise and overfitting in sparse-view synthesis.

Abstract

We present multimodal-prior-guided importance sampling as the central mechanism for hierarchical 3D Gaussian Splatting (3DGS) in sparse-view novel view synthesis. Our sampler fuses complementary cues { -- } photometric rendering residuals, semantic priors, and geometric priors { -- } to produce a robust, local recoverability estimate that directly drives where to inject fine Gaussians. Built around this sampling core, our framework comprises (1) a coarse-to-fine Gaussian representation that encodes global shape with a stable coarse layer and selectively adds fine primitives where the multimodal metric indicates recoverable detail; and (2) a geometric-aware sampling and retention policy that concentrates refinement on geometrically critical and complex regions while protecting newly added primitives in underconstrained areas from premature pruning. By prioritizing regions supported by consistent multimodal evidence rather than raw residuals alone, our method alleviates overfitting texture-induced errors and suppresses noise from pose/appearance inconsistencies. Experiments on diverse sparse-view benchmarks demonstrate state-of-the-art reconstructions, with up to +0.3 dB PSNR on DTU.