Human-centered In-building Embodied Delivery Benchmark

TL;DR

This paper introduces a new virtual environment for human-centered in-building embodied delivery, including a dataset and baseline method, to advance research in human-robot interaction for commercial scenarios.

Contribution

It presents a novel virtual environment, a large dataset of language instructions, and a baseline multimodal model for human-centered embodied delivery tasks.

Findings

Developed a multi-level connected building environment modeled after a polar research station.

Created a dataset with 13,000 language instructions for robot delivery tasks.

Proposed a baseline multimodal model for human-robot interaction in this environment.

Abstract

Recently, the concept of embodied intelligence has been widely accepted and popularized, leading people to naturally consider the potential for commercialization in this field. In this work, we propose a specific commercial scenario simulation, human-centered in-building embodied delivery. Furthermore, for this scenario, we have developed a brand-new virtual environment system from scratch, constructing a multi-level connected building space modeled after a polar research station. This environment also includes autonomous human characters and robots with grasping and mobility capabilities, as well as a large number of interactive items. Based on this environment, we have built a delivery dataset containing 13k language instructions to guide robots in providing services. We simulate human behavior through human characters and sample their various needs in daily life. Finally, we proposed a method centered around a large multimodal model to serve as the baseline system for this dataset. Compared to past embodied data work, our work focuses on a virtual environment centered around human-robot interaction for commercial scenarios. We believe this will bring new perspectives and exploration angles to the embodied community.