End-to-end differentiable design of geometric waveguide displays

TL;DR

This paper introduces a novel end-to-end differentiable framework for designing geometric waveguides in augmented reality displays, optimizing multilayer coatings and light transport for enhanced efficiency and image quality.

Contribution

It presents the first differentiable optimization method coupling Monte Carlo ray tracing with thin-film solver for waveguide design, enabling large-scale, joint optimization of optical components.

Findings

Increases light efficiency from 4.1% to 33.5%.

Improves eyebox and FoV uniformity by approximately 17x and 11x.

Automates layer pruning for topology optimization.

Abstract

Geometric waveguides are a promising architecture for optical see-through augmented reality displays, but their performance is severely bottlenecked by the difficulty of jointly optimizing non-sequential light transport and polarization-dependent multilayer thin-film coatings. Here we present the first end-to-end differentiable optimization framework for geometric waveguide that couples non-sequential Monte Carlo polarization ray tracing with a differentiable transfer-matrix thin-film solver. A differentiable Monte Carlo ray tracer avoids the exponential growth of deterministic ray splitting while enabling gradients backpropagation from eyebox metrics to design parameters. With memory-saving strategies, we optimize more than one thousand layer-thickness parameters and billions of non-sequential ray-surface intersections on a single multi-GPU workstation. Automated layer pruning is achieved by starting from over-parameterized stacks and driving redundant layers to zero thickness under discrete manufacturability constraints, effectively performing topology optimization to discover optimal coating structures. On a representative design, starting from random initialization within thickness bounds, our method increases light efficiency from 4.1\% to 33.5\% and improves eyebox and FoV uniformity by $\sim$17$\times$ and $\sim$11$\times$, respectively. Furthermore, we jointly optimize the waveguide and an image preprocessing network to improve perceived image quality. Our framework not only enables system-level, high-dimensional coating optimization inside the waveguide, but also expands the scope of differentiable optics for next-generation optical design.