Integrated Control of Robotic Arm through EMG and Speech: Decision-Driven Multimodal Data Fusion

TL;DR

This paper presents a multimodal control system for a robotic arm that integrates EMG and speech data, enabling intuitive operation through decision-driven data fusion tailored to user behavior.

Contribution

It introduces a novel multimodal data fusion approach combining EMG and speech for robotic control, with a decision-driven model that adapts to user behavior.

Findings

Effective integration of EMG and speech modalities.

Improved user interaction with robotic systems.

Adaptive decision-making enhances control accuracy.

Abstract

Interactions with electronic devices are changing in our daily lives. The day-to-day development brings curiosity to recent technology and challenges its use. The gadgets are becoming cumbersome, and their usage frustrates a segment of society. In specific scenarios, the user cannot use the modalities because of the challenges that bring in, e.g., the usage of touch screen devices by elderly people. The idea of multimodality provides easy access to devices of daily use through various modalities. In this paper, we suggest a solution that allows the operation of a microcontroller-based device using voice and speech. The model implemented will learn from the user's behavior and decide based on prior knowledge.