# Human intention recognition by deep LSTM and transformer networks for real-time human-robot collaboration

## TL;DR

This paper introduces a system that uses deep learning to recognize human intentions in real-time, improving collaboration between humans and robots in industrial settings.

## Contribution

The novel contribution is an integrated HRC system using LSTM and transformer networks for accurate intention recognition and smooth robot interaction.

## Key findings

- The system achieved higher accuracy in classifying human hand trajectories compared to previous approaches.
- Integration of dynamic movement primitives improved robot motion fluency and safety in real-world tasks.

## Abstract

Collaboration between humans and robots is essential for optimizing the performance of complex tasks in industrial environments, reducing worker strain, and improving safety. This paper presents an integrated human-robot collaboration (HRC) system that leverages advanced intention recognition for real-time task sharing and interaction. By utilizing state-of-the-art human pose estimation combined with deep learning models, we developed a robust framework for detecting and predicting worker intentions. Specifically, we employed LSTM-based and transformer-based neural networks with convolutional and pooling layers to classify human hand trajectories, achieving higher accuracy compared to previous approaches. Additionally, our system integrates dynamic movement primitives (DMPs) for smooth robot motion transitions, collision prevention, and automatic motion onset/cessation detection. We validated the system in a real-world industrial assembly task, demonstrating its effectiveness in enhancing the fluency, safety, and efficiency of human-robot collaboration. The proposed method shows promise in improving real-time decision-making in collaborative environments, offering a safer and more intuitive interaction between humans and robots.