Proximal Control of UAVs with Federated Learning for Human-Robot Collaborative Domains

TL;DR

This paper introduces a federated learning-based approach using LSTM neural networks for human action recognition and control in UAVs, addressing occlusion and distributed training challenges in multi-robot systems.

Contribution

It presents a novel federated learning framework combined with LSTM neural networks for action recognition in UAVs, enabling distributed training and improved occlusion handling.

Findings

Achieved over 96% accuracy in real-robot experiments.

Enabled distributed training without cloud dependency.

Reduced occlusion issues in multi-UAV scenarios.

Abstract

The human-robot interaction (HRI) is a growing area of research. In HRI, complex command (action) classification is still an open problem that usually prevents the real applicability of such a technique. The literature presents some works that use neural networks to detect these actions. However, occlusion is still a major issue in HRI, especially when using uncrewed aerial vehicles (UAVs), since, during the robot's movement, the human operator is often out of the robot's field of view. Furthermore, in multi-robot scenarios, distributed training is also an open problem. In this sense, this work proposes an action recognition and control approach based on Long Short-Term Memory (LSTM) Deep Neural Networks with two layers in association with three densely connected layers and Federated Learning (FL) embedded in multiple drones. The FL enabled our approach to be trained in a distributed fashion, i.e., access to data without the need for cloud or other repositories, which facilitates the multi-robot system's learning. Furthermore, our multi-robot approach results also prevented occlusion situations, with experiments with real robots achieving an accuracy greater than 96%.