Time-multiplexed Neural Holography: A flexible framework for holographic near-eye displays with fast heavily-quantized spatial light modulators

TL;DR

This paper introduces a flexible, AI-driven holography framework optimized for fast, heavily-quantized phase-only spatial light modulators, enabling high-quality near-eye displays for virtual and augmented reality.

Contribution

It develops a robust CGH optimization framework tailored for rapid, quantized SLMs, supporting diverse content types and achieving state-of-the-art results.

Findings

Effective hologram synthesis with heavily-quantized phase patterns

Supports multiple content types including RGBD and light fields

Demonstrates superior performance in simulation and experiments

Abstract

Holographic near-eye displays offer unprecedented capabilities for virtual and augmented reality systems, including perceptually important focus cues. Although artificial intelligence--driven algorithms for computer-generated holography (CGH) have recently made much progress in improving the image quality and synthesis efficiency of holograms, these algorithms are not directly applicable to emerging phase-only spatial light modulators (SLM) that are extremely fast but offer phase control with very limited precision. The speed of these SLMs offers time multiplexing capabilities, essentially enabling partially-coherent holographic display modes. Here we report advances in camera-calibrated wave propagation models for these types of holographic near-eye displays and we develop a CGH framework that robustly optimizes the heavily quantized phase patterns of fast SLMs. Our framework is flexible in supporting runtime supervision with different types of content, including 2D and 2.5D RGBD images, 3D focal stacks, and 4D light fields. Using our framework, we demonstrate state-of-the-art results for all of these scenarios in simulation and experiment.