A deep learning approach to track eye movements based on events

TL;DR

This paper presents a deep learning method using CNN_LSTM to accurately track eye movements from event camera data, aiming for cost-effective and interpretable eye tracking in VR/AR applications.

Contribution

The study introduces a novel deep learning approach combining CNN and LSTM for eye tracking from event camera inputs, achieving 81% accuracy and focusing on interpretability.

Findings

Achieved approximately 81% accuracy in locating eye centers.

Demonstrated effectiveness of CNN_LSTM model for eye movement prediction.

Proposed future work with Layer-wise Relevance Propagation for improved interpretability.

Abstract

This research project addresses the challenge of accurately tracking eye movements during specific events by leveraging previous research. Given the rapid movements of human eyes, which can reach speeds of 300{\deg}/s, precise eye tracking typically requires expensive and high-speed cameras. Our primary objective is to locate the eye center position (x, y) using inputs from an event camera. Eye movement analysis has extensive applications in consumer electronics, especially in VR and AR product development. Therefore, our ultimate goal is to develop an interpretable and cost-effective algorithm using deep learning methods to predict human attention, thereby improving device comfort and enhancing overall user experience. To achieve this goal, we explored various approaches, with the CNN\_LSTM model proving most effective, achieving approximately 81\% accuracy. Additionally, we propose future work focusing on Layer-wise Relevance Propagation (LRP) to further enhance the model's interpretability and predictive performance.