A Distributed Multi-Modal Sensing Approach for Human Activity Recognition in Real-Time Human-Robot Collaboration

TL;DR

This paper presents a real-time human activity recognition system for human-robot collaboration using a multi-modal sensing approach combining a data glove and tactile sensors, demonstrating high accuracy across various scenarios.

Contribution

It introduces a novel multi-modal sensing system integrating a data glove and tactile sensors for improved real-time human activity recognition in HRC.

Findings

High accuracy in offline and real-time classification

Effective in static and collaborative scenarios

Multi-modal approach benefits HRC tasks

Abstract

Human activity recognition (HAR) is fundamental in human-robot collaboration (HRC), enabling robots to respond to and dynamically adapt to human intentions. This paper introduces a HAR system combining a modular data glove equipped with Inertial Measurement Units and a vision-based tactile sensor to capture hand activities in contact with a robot. We tested our activity recognition approach under different conditions, including offline classification of segmented sequences, real-time classification under static conditions, and a realistic HRC scenario. The experimental results show a high accuracy for all the tasks, suggesting that multiple collaborative settings could benefit from this multi-modal approach.