Humanizing Robot Gaze Shifts: A Framework for Natural Gaze Shifts in Humanoid Robots

TL;DR

This paper presents the Robot Gaze-Shift framework that combines gaze reasoning and biomimetic motion generation to enable humanoid robots to perform natural, context-aware gaze shifts during social interactions.

Contribution

It introduces a unified pipeline integrating multimodal gaze reasoning with a VQ-VAE-based motion generator for realistic gaze behaviors in robots.

Findings

RGS replicates human-like gaze target selection

Generates diverse, realistic gaze-shift motions

Effective in unconstrained human-robot interactions

Abstract

Leveraging auditory and visual feedback for attention reorientation is essential for natural gaze shifts in social interaction. However, enabling humanoid robots to perform natural and context-appropriate gaze shifts in unconstrained human--robot interaction (HRI) remains challenging, as it requires the coupling of cognitive attention mechanisms and biomimetic motion generation. In this work, we propose the Robot Gaze-Shift (RGS) framework, which integrates these two components into a unified pipeline. First, RGS employs a vision--language model (VLM)-based gaze reasoning pipeline to infer context-appropriate gaze targets from multimodal interaction cues, ensuring consistency with human gaze-orienting regularities. Second, RGS introduces a conditional Vector Quantized-Variational Autoencoder (VQ-VAE) model for eye--head coordinated gaze-shift motion generation, producing diverse and human-like gaze-shift behaviors. Experiments validate that RGS effectively replicates human-like target selection and generates realistic, diverse gaze-shift motions.