A Data-driven Approach Towards Human-robot Collaborative Problem Solving in a Shared Space

TL;DR

This paper presents a data-driven system for natural human-robot communication in shared problem-solving environments, utilizing multiple data sources and machine learning to improve interaction and collaboration.

Contribution

It introduces a multi-stage, data-driven framework for developing human-robot communication systems in shared spaces, including data collection tools and initial evaluation methods.

Findings

Collected initial communication data from human pairs

Developed a web application and mobile laboratory for data gathering

Established performance metrics for system evaluation

Abstract

We are developing a system for human-robot communication that enables people to communicate with robots in a natural way and is focused on solving problems in a shared space. Our strategy for developing this system is fundamentally data-driven: we use data from multiple input sources and train key components with various machine learning techniques. We developed a web application that is collecting data on how two humans communicate to accomplish a task, as well as a mobile laboratory that is instrumented to collect data on how two humans communicate to accomplish a task in a physically shared space. The data from these systems will be used to train and fine-tune the second stage of our system, in which the robot will be simulated through software. A physical robot will be used in the final stage of our project. We describe these instruments, a test-suite and performance metrics designed to evaluate and automate the data gathering process as well as evaluate an initial data set.