IDESplat: Iterative Depth Probability Estimation for Generalizable 3D Gaussian Splatting

TL;DR

IDESplat introduces an iterative depth estimation method using warp operations and epipolar attention to improve 3D Gaussian Splatting, achieving state-of-the-art results with real-time efficiency and strong generalization.

Contribution

It proposes a novel iterative depth boosting framework with a Depth Probability Boosting Unit for more accurate Gaussian mean prediction in 3D scene reconstruction.

Findings

Outperforms DepthSplat in PSNR by 0.33 dB on RE10K

Achieves 2.95 dB higher PSNR on DTU dataset

Uses significantly fewer parameters and memory

Abstract

Generalizable 3D Gaussian Splatting aims to directly predict Gaussian parameters using a feed-forward network for scene reconstruction. Among these parameters, Gaussian means are particularly difficult to predict, so depth is usually estimated first and then unprojected to obtain the Gaussian sphere centers. Existing methods typically rely solely on a single warp to estimate depth probability, which hinders their ability to fully leverage cross-view geometric cues, resulting in unstable and coarse depth maps. To address this limitation, we propose IDESplat, which iteratively applies warp operations to boost depth probability estimation for accurate Gaussian mean prediction. First, to eliminate the inherent instability of a single warp, we introduce a Depth Probability Boosting Unit (DPBU) that integrates epipolar attention maps produced by cascading warp operations in a multiplicative manner. Next, we construct an iterative depth estimation process by stacking multiple DPBUs, progressively identifying potential depth candidates with high likelihood. As IDESplat iteratively boosts depth probability estimates and updates the depth candidates, the depth map is gradually refined, resulting in accurate Gaussian means. We conduct experiments on RealEstate10K, ACID, and DL3DV. IDESplat achieves outstanding reconstruction quality and state-of-the-art performance with real-time efficiency. On RE10K, it outperforms DepthSplat by 0.33 dB in PSNR, using only 10.7% of the parameters and 70% of the memory. Additionally, our IDESplat improves PSNR by 2.95 dB over DepthSplat on the DTU dataset in cross-dataset experiments, demonstrating its strong generalization ability.