Selective Eye-gaze Augmentation To Enhance Imitation Learning In Atari Games
Chaitanya Thammineni, Hemanth Manjunatha, Ehsan T. Esfahani

TL;DR
This paper introduces a selective eye-gaze augmentation network that improves imitation learning in Atari games by intelligently deciding when to incorporate eye-gaze data, outperforming existing methods.
Contribution
The paper proposes a novel SEA network that learns to selectively use eye-gaze information, enhancing imitation learning performance in Atari games.
Findings
SEA outperforms state-of-the-art attention-guided imitation learning and behavior cloning.
Selective gaze use significantly improves learning compared to random gaze selection.
The gating mechanism effectively determines when to incorporate eye-gaze data.
Abstract
This paper presents the selective use of eye-gaze information in learning human actions in Atari games. Vast evidence suggests that our eye movement convey a wealth of information about the direction of our attention and mental states and encode the information necessary to complete a task. Based on this evidence, we hypothesize that selective use of eye-gaze, as a clue for attention direction, will enhance the learning from demonstration. For this purpose, we propose a selective eye-gaze augmentation (SEA) network that learns when to use the eye-gaze information. The proposed network architecture consists of three sub-networks: gaze prediction, gating, and action prediction network. Using the prior 4 game frames, a gaze map is predicted by the gaze prediction network which is used for augmenting the input frame. The gating network will determine whether the predicted gaze map should be…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
