Proactive Interaction Framework for Intelligent Social Receptionist Robots

TL;DR

This paper introduces TFVT-HRI, an end-to-end transformer-based framework that enhances proactive human-robot interaction for reception robots by accurately interpreting scenarios and selecting from over 1000 diverse actions, improving social acceptability.

Contribution

The paper presents a novel transformer-based visual token approach for proactive HRI, enabling more nuanced and diverse robot behaviors compared to existing rule-based or limited end-to-end models.

Findings

Achieves state-of-the-art performance in action triggering and selection.

Demonstrates increased humanness and intelligence in real-world office environments.

Handles over 1000 diverse proactive behaviors.

Abstract

Proactive human-robot interaction (HRI) allows the receptionist robots to actively greet people and offer services based on vision, which has been found to improve acceptability and customer satisfaction. Existing approaches are either based on multi-stage decision processes or based on end-to-end decision models. However, the rule-based approaches require sedulous expert efforts and only handle minimal pre-defined scenarios. On the other hand, existing works with end-to-end models are limited to very general greetings or few behavior patterns (typically less than 10). To address those challenges, we propose a new end-to-end framework, the TransFormer with Visual Tokens for Human-Robot Interaction (TFVT-HRI). The proposed framework extracts visual tokens of relative objects from an RGB camera first. To ensure the correct interpretation of the scenario, a transformer decision model is then employed to process the visual tokens, which is augmented with the temporal and spatial information. It predicts the appropriate action to take in each scenario and identifies the right target. Our data is collected from an in-service receptionist robot in an office building, which is then annotated by experts for appropriate proactive behavior. The action set includes 1000+ diverse patterns by combining language, emoji expression, and body motions. We compare our model with other SOTA end-to-end models on both offline test sets and online user experiments in realistic office building environments to validate this framework. It is demonstrated that the decision model achieves SOTA performance in action triggering and selection, resulting in more humanness and intelligence when compared with the previous reactive reception policies.