Time-Division Multiplexing Light Field Display with Learned Coded Aperture

TL;DR

This paper introduces a deep learning-based coded time-division multiplexing method for light field displays, improving resolution and reducing aliasing by jointly optimizing sub-aperture views and aperture patterns.

Contribution

It presents a novel deep learning framework that optimizes light field display parameters end-to-end, enhancing image quality and addressing resolution trade-offs in integral-imaging-based displays.

Findings

Higher PSNR, SSIM, and LPIPS scores compared to baseline methods.

Effective joint optimization of sub-aperture views and aperture patterns.

Improved perceived image quality in light field displays.

Abstract

Conventional stereoscopic displays suffer from vergence-accommodation conflict and cause visual fatigue. Integral-imaging-based displays resolve the problem by directly projecting the sub-aperture views of a light field into the eyes using a microlens array or a similar structure. However, such displays have an inherent trade-off between angular and spatial resolutions. In this paper, we propose a novel coded time-division multiplexing technique that projects encoded sub-aperture views to the eyes of a viewer with correct cues for vergence-accommodation reflex. Given sparse light field sub-aperture views, our pipeline can provide a perception of high-resolution refocused images with minimal aliasing by jointly optimizing the sub-aperture views for display and the coded aperture pattern. This is achieved via deep learning in an end-to-end fashion by simulating light transport and image formation with Fourier optics. To our knowledge, this work is among the first that optimize the light field display pipeline with deep learning. We verify our idea with objective image quality metrics (PSNR, SSIM, and LPIPS) and perform an extensive study on various customizable design variables in our display pipeline. Experimental results show that light fields displayed using the proposed technique indeed have higher quality than that of baseline display designs.