Real-time 3D Light-field Viewing with Eye-tracking on Conventional Displays

TL;DR

This paper presents a cost-effective, real-time 3D light-field viewing system that uses standard hardware and eye-tracking to deliver immersive experiences without specialized equipment.

Contribution

It introduces a novel method for dynamic 3D light-field visualization on conventional displays using eye-tracking and pre-captured light-field data.

Findings

Operates at 30 FPS on CPU

Uses standard monitor and webcam

Provides immersive 3D experience

Abstract

Creating immersive 3D visual experiences typically requires expensive and specialized hardware such as VR headsets, autostereoscopic displays, or active shutter glasses. These constraints limit the accessibility and everyday use of 3D visualization technologies in resource-constrained settings. To address this, we propose a low-cost system that enables real-time 3D light-field viewing using only a standard 2D monitor, a conventional RGB webcam, and red-cyan anaglyph glasses. The system integrates real-time eye-tracking to dynamically adapt the displayed light-field image to the user's head position with a lightweight rendering pipeline that selects and composites stereoscopic views from pre-captured light-field data. The resulting anaglyph image is updated in real-time, creating a more immersive and responsive 3D experience. The system operates entirely on CPU and maintains a stable frame rate of 30 FPS, confirming its feasibility on typical consumer-grade hardware. All of these highlight the potential of our approach as an accessible platform for interactive 3D applications in education, digital media, and beyond.