Developing Neural Network-Based Gaze Control Systems for Social Robots

TL;DR

This paper develops neural network models to predict human gaze behavior in social interactions and implements the best model on a robot, enhancing social engagement capabilities.

Contribution

It introduces a novel deep learning approach using LSTM and Transformers to model and predict gaze patterns in social contexts for robots.

Findings

Models achieved 60-65% accuracy in gaze prediction.

Participants rated the robot's gaze behavior positively.

Experienced users rated the system more favorably.

Abstract

During multi-party interactions, gaze direction is a key indicator of interest and intent, making it essential for social robots to direct their attention appropriately. Understanding the social context is crucial for robots to engage effectively, predict human intentions, and navigate interactions smoothly. This study aims to develop an empirical motion-time pattern for human gaze behavior in various social situations (e.g., entering, leaving, waving, talking, and pointing) using deep neural networks based on participants' data. We created two video clips-one for a computer screen and another for a virtual reality headset-depicting different social scenarios. Data were collected from 30 participants: 15 using an eye-tracker and 15 using an Oculus Quest 1 headset. Deep learning models, specifically Long Short-Term Memory (LSTM) and Transformers, were used to analyze and predict gaze patterns. Our models achieved 60% accuracy in predicting gaze direction in a 2D animation and 65% accuracy in a 3D animation. Then, the best model was implemented onto the Nao robot; and 36 new participants evaluated its performance. The feedback indicated overall satisfaction, with those experienced in robotics rating the models more favorably.