Rendering of Eyes for Eye-Shape Registration and Gaze Estimation

TL;DR

This paper introduces SynthesEyes, a method for generating highly realistic, labeled eye images using computer graphics, which improves the training of eye-shape registration and gaze estimation models.

Contribution

It presents a novel approach to synthesize photo-realistic eye images with accurate labels, reducing data collection time and enhancing model performance.

Findings

SynthesEyes outperforms state-of-the-art methods in eye-shape registration.

SynthesEyes improves cross-dataset gaze estimation accuracy.

Realistic illumination and shape variations are crucial for training data quality.

Abstract

Images of the eye are key in several computer vision problems, such as shape registration and gaze estimation. Recent large-scale supervised methods for these problems require time-consuming data collection and manual annotation, which can be unreliable. We propose synthesizing perfectly labelled photo-realistic training data in a fraction of the time. We used computer graphics techniques to build a collection of dynamic eye-region models from head scan geometry. These were randomly posed to synthesize close-up eye images for a wide range of head poses, gaze directions, and illumination conditions. We used our model's controllability to verify the importance of realistic illumination and shape variations in eye-region training data. Finally, we demonstrate the benefits of our synthesized training data (SynthesEyes) by out-performing state-of-the-art methods for eye-shape registration as well as cross-dataset appearance-based gaze estimation in the wild.