Dance2Hesitate: A Multi-Modal Dataset of Dancer-Taught Hesitancy for Understandable Robot Motion

TL;DR

This paper introduces Dance2Hesitate, a comprehensive multi-modal dataset of dancer-generated and robot kinesthetic demonstrations capturing varying levels of hesitancy, to improve robot motion understanding and expression in collaborative tasks.

Contribution

The paper presents a novel, open-source multi-modal dataset of hesitant motions from dancers and robots, enabling better generalization and benchmarking of hesitant robot behaviors.

Findings

Collected 70 whole-body trajectories across hesitancy levels.

Captured 84 upper limb trajectories with graded hesitancy.

Provided 66 kinesthetic teaching trajectories for robot motion.

Abstract

In human-robot collaboration, a robot's expression of hesitancy is a critical factor that shapes human coordination strategies, attention allocation, and safety-related judgments. However, designing hesitant robot motion that generalizes is challenging because the observer's inference is highly dependent on embodiment and context. To address these challenges, we introduce and open-source a multi-modal, dancer-generated dataset of hesitant motion where we focus on specific context-embodiment pairs (i.e., manipulator/human upper-limb approaching a Jenga Tower, and anthropomorphic whole body motion in free space). The dataset includes (i) kinesthetic teaching demonstrations on a Franka Emika Panda reaching from a fixed start configuration to a fixed target (a Jenga tower) with three graded hesitancy levels (slight, significant, extreme) and (ii) synchronized RGB-D motion capture of dancers performing the same reaching behavior using their upper limb across three hesitancy levels, plus full human body sequences for extreme hesitancy. We further provide documentation to enable reproducible benchmarking across robot and human modalities. Across all dancers, we obtained 70 unique whole-body trajectories, 84 upper limb trajectories spanning over the three hesitancy levels, and 66 kinesthetic teaching trajectories spanning over the three hesitancy levels. The dataset can be accessed here: https://brsrikrishna.github.io/Dance2Hesitate/.