Bidirectional Human-Robot Communication for Physical Human-Robot Interaction

TL;DR

This paper presents BRIDGE, a bidirectional communication system enabling users to modify robot trajectories via natural language and receive verbal feedback, enhancing transparency and interactivity in physical human-robot interaction.

Contribution

Introduces BRIDGE, a novel system that uses large language models for real-time trajectory modification and verbal feedback in physical human-robot interaction.

Findings

Participants successfully modified robot trajectories using natural language.

Verbal feedback increased perceived interactivity and transparency.

System improved user experience compared to non-verbal feedback baseline.

Abstract

Effective physical human-robot interaction requires systems that are not only adaptable to user preferences but also transparent about their actions. This paper introduces BRIDGE, a system for bidirectional human-robot communication in physical assistance. Our method allows users to modify a robot's planned trajectory -- position, velocity, and force -- in real time using natural language. We utilize a large language model (LLM) to interpret any trajectory modifications implied by user commands in the context of the planned motion and conversation history. Importantly, our system provides verbal feedback in response to the user, either assuring any resulting changes or posing a clarifying question. We evaluated our method in a user study with 18 older adults across three assistive tasks, comparing BRIDGE to an ablation without verbal feedback and a baseline. Results show that participants successfully used the system to modify trajectories in real time. Moreover, the bidirectional feedback led to significantly higher ratings of interactivity and transparency, demonstrating that the robot's verbal response is critical for a more intuitive user experience. Videos and code can be found on our project website: https://bidir-comm.github.io/