Real-time 3D Visualization of Radiance Fields on Light Field Displays

TL;DR

This paper introduces a unified, real-time rendering framework for radiance fields on light field displays, enabling high-quality, immersive 3D visualization at over 200 FPS across multiple views.

Contribution

It presents a novel, efficient architecture supporting various radiance field representations for real-time light field display rendering without retraining.

Findings

Achieves 200+ FPS at 512p on a Looking Glass display.

Provides up to 22x speedup over independent view rendering.

Maintains high image quality across different scene formats.

Abstract

Radiance fields have revolutionized photo-realistic 3D scene visualization by enabling high-fidelity reconstruction of complex environments, making them an ideal match for light field displays. However, integrating these technologies presents significant computational challenges, as light field displays require multiple high-resolution renderings from slightly shifted viewpoints, while radiance fields rely on computationally intensive volume rendering. In this paper, we propose a unified and efficient framework for real-time radiance field rendering on light field displays. Our method supports a wide range of radiance field representations, including NeRFs, 3D Gaussian Splatting, and Sparse Voxels, within a shared architecture based on a single-pass plane sweeping strategy and caching of shared, non-directional components. The framework generalizes across different scene formats without retraining, and avoids redundant computation across views. We further demonstrate a real-time interactive application on a Looking Glass display, achieving 200+ FPS at 512p across 45 views, enabling seamless, immersive 3D interaction. On standard benchmarks, our method achieves up to 22x speedup compared to independently rendering each view, while preserving image quality.