That and There: Judging the Intent of Pointing Actions with Robotic Arms

TL;DR

This paper develops interpretive principles for robotic pointing actions to improve human-robot communication, evaluating how humans interpret different types of pointing in task instructions through animations.

Contribution

It introduces a set of principles for robotic pointing communication and empirically evaluates their effectiveness in human interpretation during simulated tasks.

Findings

Humans interpret referential pointing more flexibly than locating pointing.

Environmental and contextual variations influence pointing interpretation.

Referential pointing is easier for humans to understand than locating pointing.

Abstract

Collaborative robotics requires effective communication between a robot and a human partner. This work proposes a set of interpretive principles for how a robotic arm can use pointing actions to communicate task information to people by extending existing models from the related literature. These principles are evaluated through studies where English-speaking human subjects view animations of simulated robots instructing pick-and-place tasks. The evaluation distinguishes two classes of pointing actions that arise in pick-and-place tasks: referential pointing (identifying objects) and locating pointing (identifying locations). The study indicates that human subjects show greater flexibility in interpreting the intent of referential pointing compared to locating pointing, which needs to be more deliberate. The results also demonstrate the effects of variation in the environment and task context on the interpretation of pointing. Our corpus, experiments and design principles advance models of context, common sense reasoning and communication in embodied communication.