Realtime Dynamic Gaze Target Tracking and Depth-Level Estimation

TL;DR

This paper introduces a real-time gaze tracking and depth estimation system for transparent displays, combining a tree-based target tracking algorithm with a self-attention model for depth inference, validated on real-world data and hardware.

Contribution

It presents a novel integrated approach for real-time gaze target tracking and depth-level estimation on dynamic transparent displays, addressing key challenges in human-device interaction.

Findings

High precision and scalability demonstrated on real-world datasets.

Real-time performance validated on System on Chip (SoC) hardware.

Effective handling of dynamic, overlapping, and size-changing gaze targets.

Abstract

The integration of Transparent Displays (TD) in various applications, such as Heads-Up Displays (HUDs) in vehicles, is a burgeoning field, poised to revolutionize user experiences. However, this innovation brings forth significant challenges in realtime human-device interaction, particularly in accurately identifying and tracking a user's gaze on dynamically changing TDs. In this paper, we present a two-fold robust and efficient systematic solution for realtime gaze monitoring, comprised of: (1) a tree-based algorithm for identifying and dynamically tracking gaze targets (i.e., moving, size-changing, and overlapping 2D content) projected on a transparent display, in realtime; (2) a multi-stream self-attention architecture to estimate the depth-level of human gaze from eye tracking data, to account for the display's transparency and preventing undesired interactions with the TD. We collected a real-world eye-tracking dataset to train and test our gaze monitoring system. We present extensive results and ablation studies, including inference experiments on System on Chip (SoC) evaluation boards, demonstrating our model's scalability, precision, and realtime feasibility in both static and dynamic contexts. Our solution marks a significant stride in enhancing next-generation user-device interaction and experience, setting a new benchmark for algorithmic gaze monitoring technology in dynamic transparent displays.