Gesture2Path: Imitation Learning for Gesture-aware Navigation

TL;DR

Gesture2Path enables robots to interpret human gestures through neural networks and generate socially compliant navigation paths, improving human-robot interaction in crowded environments.

Contribution

It introduces a novel approach combining image-based imitation learning with model predictive control for gesture-aware social navigation.

Findings

Successfully interpreted complex gestures like left/right, follow me, and make a circle.

Generated socially compliant trajectories in real-world robot experiments.

Validated effectiveness through participant ratings in interactive scenarios.

Abstract

As robots increasingly enter human-centered environments, they must not only be able to navigate safely around humans, but also adhere to complex social norms. Humans often rely on non-verbal communication through gestures and facial expressions when navigating around other people, especially in densely occupied spaces. Consequently, robots also need to be able to interpret gestures as part of solving social navigation tasks. To this end, we present Gesture2Path, a novel social navigation approach that combines image-based imitation learning with model-predictive control. Gestures are interpreted based on a neural network that operates on streams of images, while we use a state-of-the-art model predictive control algorithm to solve point-to-point navigation tasks. We deploy our method on real robots and showcase the effectiveness of our approach for the four gestures-navigation scenarios: left/right, follow me, and make a circle. Our experiments indicate that our method is able to successfully interpret complex human gestures and to use them as a signal to generate socially compliant trajectories for navigation tasks. We validated our method based on in-situ ratings of participants interacting with the robots.