Human Arm Pose Estimation with a Shoulder-worn Force-Myography Device for Human-Robot Interaction

TL;DR

This paper introduces a shoulder-worn force-myography device combined with a Transformer model to estimate human arm pose for real-time human-robot interaction, enabling collision avoidance without visual sensors.

Contribution

It presents a novel application of FMG for full arm pose estimation and demonstrates a transferable, real-time model for HRI tasks.

Findings

Effective arm pose estimation using FMG data

Model transferability across different users

Successful collision avoidance in real-world tests

Abstract

Accurate human pose estimation is essential for effective Human-Robot Interaction (HRI). By observing a user's arm movements, robots can respond appropriately, whether it's providing assistance or avoiding collisions. While visual perception offers potential for human pose estimation, it can be hindered by factors like poor lighting or occlusions. Additionally, wearable inertial sensors, though useful, require frequent calibration as they do not provide absolute position information. Force-myography (FMG) is an alternative approach where muscle perturbations are externally measured. It has been used to observe finger movements, but its application to full arm state estimation is unexplored. In this letter, we investigate the use of a wearable FMG device that can observe the state of the human arm for real-time applications of HRI. We propose a Transformer-based model to map FMG measurements from the shoulder of the user to the physical pose of the arm. The model is also shown to be transferable to other users with limited decline in accuracy. Through real-world experiments with a robotic arm, we demonstrate collision avoidance without relying on visual perception.