SOCRATES: Text-based Human Search and Approach using a Robot Dog

TL;DR

This paper introduces SOCRATES, a model enabling robots to search for and approach humans based on free-text descriptions, combining language models with motion generation, validated through simulations and real-world experiments.

Contribution

It presents a novel Socratic model for human search and approach tasks that integrates large language models with motion generation modules for robots.

Findings

Effective human search using textual descriptions demonstrated in simulations and real-world tests.

Generated robot approach motions are human-friendly and validated with human participant feedback.

The framework improves robot-human interaction quality based on RoSAS analysis.

Abstract

In this paper, we propose a SOCratic model for Robots Approaching humans based on TExt System (SOCRATES) focusing on the human search and approach based on free-form textual description; the robot first searches for the target user, then the robot proceeds to approach in a human-friendly manner. In particular, textual descriptions are composed of appearance (e.g., wearing white shirts with black hair) and location clues (e.g., is a student who works with robots). We initially present a Human Search Socratic Model that connects large pre-trained models in the language domain to solve the downstream task, which is searching for the target person based on textual descriptions. Then, we propose a hybrid learning-based framework for generating target-cordial robotic motion to approach a person, consisting of a learning-from-demonstration module and a knowledge distillation module. We validate the proposed searching module via simulation using a virtual mobile robot as well as through real-world experiments involving participants and the Boston Dynamics Spot robot. Furthermore, we analyze the properties of the proposed approaching framework with human participants based on the Robotic Social Attributes Scale (RoSAS)