# Utilizing Eye Gaze to Enhance the Generalization of Imitation Networks to Unseen Environments

## TL;DR

This paper demonstrates that incorporating gaze behavior into imitation learning models significantly improves their ability to generalize to unseen driving environments, leveraging additional human cues beyond raw images and actions.

## Contribution

The study introduces methods for integrating gaze data into imitation networks, showing improved generalization in autonomous driving tasks.

## Key findings

- Gaze information enhances model generalization to new environments.
- Different approaches to integrating gaze data are effective.
- Gaze cues provide valuable supplementary information for imitation learning.

## Abstract

Vision-based autonomous driving through imitation learning mimics the behaviors of human drivers by training on pairs of data of raw driver-view images and actions. However, there are other cues, e.g. gaze behavior, available from human drivers that have yet to be exploited. Previous research has shown that novice human learners can benefit from observing experts' gaze patterns. We show here that deep neural networks can also benefit from this. We demonstrate different approaches to integrating gaze information into imitation networks. Our results show that the integration of gaze information improves the generalization performance of networks to unseen environments.