RIT-Eyes: Rendering of near-eye images for eye-tracking applications

TL;DR

This paper introduces RIT-Eyes, a synthetic eye image generation platform with advanced features to improve training data for video-based eye tracking neural networks, reducing manual labeling efforts and enhancing model robustness.

Contribution

The work presents a novel synthetic eye image generator with realistic eye features, enabling better training of eye-tracking models and demonstrating its effectiveness on public datasets.

Findings

Rendered images reflect real gaze distributions

Models trained on synthetic data perform well on real datasets

Enhanced features improve training data realism

Abstract

Deep neural networks for video-based eye tracking have demonstrated resilience to noisy environments, stray reflections, and low resolution. However, to train these networks, a large number of manually annotated images are required. To alleviate the cumbersome process of manual labeling, computer graphics rendering is employed to automatically generate a large corpus of annotated eye images under various conditions. In this work, we introduce a synthetic eye image generation platform that improves upon previous work by adding features such as an active deformable iris, an aspherical cornea, retinal retro-reflection, gaze-coordinated eye-lid deformations, and blinks. To demonstrate the utility of our platform, we render images reflecting the represented gaze distributions inherent in two publicly available datasets, NVGaze and OpenEDS. We also report on the performance of two semantic segmentation architectures (SegNet and RITnet) trained on rendered images and tested on the original datasets.