Shared Affordance-awareness via Augmented Reality for Proactive Assistance in Human-robot Collaboration

TL;DR

This paper proposes an augmented reality framework to enhance human-robot collaboration by sharing affordance-awareness, enabling proactive assistance through shared perception and communication in domestic tasks.

Contribution

It introduces an AR-based shared affordance-awareness system that improves communication and proactive assistance in human-robot collaboration.

Findings

Framework effectively models shared environment and affordance regions.

AR interface facilitates communication of affordances and assistance prompts.

Potential to improve autonomous task performance in assistive robots.

Abstract

Enabling humans and robots to collaborate effectively requires purposeful communication and an understanding of each other's affordances. Prior work in human-robot collaboration has incorporated knowledge of human affordances, i.e., their action possibilities in the current context, into autonomous robot decision-making. This "affordance awareness" is especially promising for service robots that need to know when and how to assist a person that cannot independently complete a task. However, robots still fall short in performing many common tasks autonomously. In this work-in-progress paper, we propose an augmented reality (AR) framework that bridges the gap in an assistive robot's capabilities by actively engaging with a human through a shared affordance-awareness representation. Leveraging the different perspectives from a human wearing an AR headset and a robot's equipped sensors, we can build a perceptual representation of the shared environment and model regions of respective agent affordances. The AR interface can also allow both agents to communicate affordances with one another, as well as prompt for assistance when attempting to perform an action outside their affordance region. This paper presents the main components of the proposed framework and discusses its potential through a domestic cleaning task experiment.