Where Do We Look When We Teach? Analyzing Human Gaze Behavior Across Demonstration Devices in Robot Imitation Learning

TL;DR

This paper investigates how different demonstration devices affect human gaze behavior in robot imitation learning, revealing that more natural devices enhance task success rates significantly under environmental changes.

Contribution

It systematically analyzes the impact of various demonstration devices on gaze behavior and demonstrates that natural human-like devices improve imitation learning outcomes.

Findings

Devices emulating robot embodiment impair gaze cue extraction

Devices mimicking visual conditions also impair gaze behavior

Natural human-like gaze data increases task success rate from 18.8% to 68.8%

Abstract

Imitation learning for acquiring generalizable policies often requires a large volume of demonstration data, making the process significantly costly. One promising strategy to address this challenge is to leverage the cognitive and decision-making skills of human demonstrators with strong generalization capability, particularly by extracting task-relevant cues from their gaze behavior. However, imitation learning typically involves humans collecting data using demonstration devices that emulate a robot's embodiment and visual condition. This raises the question of how such devices influence gaze behavior. We propose an experimental framework that systematically analyzes demonstrators' gaze behavior across a spectrum of demonstration devices. Our experimental results indicate that devices emulating (1) a robot's embodiment or (2) visual condition impair demonstrators' capability to extract task-relevant cues via gaze behavior, with the extent of impairment depending on the degree of emulation. Additionally, gaze data collected using devices that capture natural human behavior improves the policy's task success rate from 18.8% to 68.8% under environmental shifts.