Weakly-Supervised Degree of Eye-Closeness Estimation

TL;DR

This paper introduces a weakly-supervised approach for estimating the degree of eye closeness, leveraging synthetic data with detailed annotations and real data with weak labels to improve accuracy in real-world applications.

Contribution

The paper presents a novel weakly-supervised method that combines synthetic detailed annotations with real weak labels to accurately estimate eye openness levels.

Findings

Effective domain adaptation from synthetic to real data

High accuracy in degree of eye closeness estimation

Large-scale synthetic and real datasets used for training

Abstract

Following recent technological advances there is a growing interest in building non-intrusive methods that help us communicate with computing devices. In this regard, accurate information from eye is a promising input medium between a user and computing devices. In this paper we propose a method that captures the degree of eye closeness. Although many methods exist for detection of eyelid openness, they are inherently unable to satisfactorily perform in real world applications. Detailed eye state estimation is more important, in extracting meaningful information, than estimating whether eyes are open or closed. However, learning reliable eye state estimator requires accurate annotations which is cost prohibitive. In this work, we leverage synthetic face images which can be generated via computer graphics rendering techniques and automatically annotated with different levels of eye openness. These synthesized training data images, however, have a domain shift from real-world data. To alleviate this issue, we propose a weakly-supervised method which utilizes the accurate annotation from the synthetic data set, to learn accurate degree of eye openness, and the weakly labeled (open or closed) real world eye data set to control the domain shift. We introduce a data set of 1.3M synthetic face images with detail eye openness and eye gaze information, and 21k real-world images with open/closed annotation. The dataset will be released online upon acceptance. Extensive experiments validate the effectiveness of the proposed approach.