Eye-gaze Estimation with HEOG and Neck EMG using Deep Neural Networks

TL;DR

This study demonstrates that deep neural networks can effectively estimate eye-gaze by combining HEOG and neck EMG signals, improving accuracy over traditional methods, especially in complex head movement scenarios.

Contribution

The paper introduces a novel approach using deep neural networks to combine HEOG and NEMG signals for reliable eye-gaze estimation, surpassing traditional feature-based methods.

Findings

DNN with HEOG and IMU achieved 93.3% accuracy.

Combining HEOG with NEMG improved accuracy to 72.6%.

The method outperforms using HEOG or NEMG alone.

Abstract

Hearing-impaired listeners usually have troubles attending target talker in multi-talker scenes, even with hearing aids (HAs). The problem can be solved with eye-gaze steering HAs, which requires listeners eye-gazing on the target. In a situation where head rotates, eye-gaze is subject to both behaviors of saccade and head rotation. However, existing methods of eye-gaze estimation did not work reliably, since the listener's strategy of eye-gaze varies and measurements of the two behaviors were not properly combined. Besides, existing methods were based on hand-craft features, which could overlook some important information. In this paper, a head-fixed and a head-free experiments were conducted. We used horizontal electrooculography (HEOG) and neck electromyography (NEMG), which separately measured saccade and head rotation to commonly estimate eye-gaze. Besides traditional classifier and hand-craft features, deep neural networks (DNN) were introduced to automatically extract features from intact waveforms. Evaluation results showed that when the input was HEOG with inertial measurement unit, the best performance of our proposed DNN classifiers achieved 93.3%; and when HEOG was with NEMG together, the accuracy reached 72.6%, higher than that with HEOG (about 71.0%) or NEMG (about 35.7%) alone. These results indicated the feasibility to estimate eye-gaze with HEOG and NEMG.