Hybrid Foveated Path Tracing with Peripheral Gaussians for Immersive Anatomy

TL;DR

This paper introduces a hybrid rendering method combining foveated path tracing with Gaussian Splatting to enable high-quality, interactive, and immersive 3D medical visualization without extensive preprocessing.

Contribution

The authors develop a novel hybrid approach that integrates streamed foveated path tracing with Gaussian Splatting, allowing real-time updates and improved peripheral visualization in medical imaging.

Findings

The hybrid method maintains high visual quality comparable to direct path tracing.

Peripheral models can be re-generated in under a second, enabling interactive use.

The approach reduces preprocessing time compared to traditional Gaussian Splatting.

Abstract

Volumetric medical imaging offers great potential for understanding complex pathologies. Yet, traditional 2D slices provide little support for interpreting spatial relationships, forcing users to mentally reconstruct anatomy into three dimensions. Direct volumetric path tracing and VR rendering can improve perception but are computationally expensive, while precomputed representations, like Gaussian Splatting, require planning ahead. Both approaches limit interactive use. We propose a hybrid rendering approach for high-quality, interactive, and immersive anatomical visualization. Our method combines streamed foveated path tracing with a lightweight Gaussian Splatting approximation of the periphery. The peripheral model generation is optimized with volume data and continuously refined using foveal renderings, enabling interactive updates. Depth-guided reprojection further improves robustness to latency and allows users to balance fidelity with refresh rate. We compare our method against direct path tracing and Gaussian Splatting. Our results highlight how their combination can preserve strengths in visual quality while re-generating the peripheral model in under a second, eliminating extensive preprocessing and approximations. This opens new options for interactive medical visualization.