Assessing Similarity Measures for the Evaluation of Human-Robot Motion Correspondence

TL;DR

This paper proposes using heterogeneous time-series similarity measures, especially Gromov Dynamic Time Warping, as a quantitative way to evaluate human-robot motion correspondence, complementing traditional qualitative surveys.

Contribution

It introduces a quantitative evaluation method for motion correspondence using similarity measures and validates Gromov Dynamic Time Warping as an effective metric.

Findings

Gromov Dynamic Time Warping correlates well with human survey results.

Quantitative measures can complement qualitative surveys in evaluating motion correspondence.

The proposed approach offers a more scalable evaluation method.

Abstract

One key area of research in Human-Robot Interaction is solving the human-robot correspondence problem, which asks how a robot can learn to reproduce a human motion demonstration when the human and robot have different dynamics and kinematic structures. Evaluating these correspondence problem solutions often requires the use of qualitative surveys that can be time consuming to design and administer. Additionally, qualitative survey results vary depending on the population of survey participants. In this paper, we propose the use of heterogeneous time-series similarity measures as a quantitative evaluation metric for evaluating motion correspondence to complement these qualitative surveys. To assess the suitability of these measures, we develop a behavioral cloning-based motion correspondence model, and evaluate it with a qualitative survey as well as quantitative measures. By comparing the resulting similarity scores with the human survey results, we identify Gromov Dynamic Time Warping as a promising quantitative measure for evaluating motion correspondence.