Unimodal and Multimodal Sensor Fusion for Wearable Activity Recognition

TL;DR

This paper explores the integration of multiple sensing modalities and machine learning algorithms in wearable devices to improve human activity recognition across various scenarios, emphasizing real-time embedded processing.

Contribution

It introduces a multidisciplinary approach combining inertial, pressure, and textile sensors with machine learning for enhanced wearable activity recognition.

Findings

Successful real-time implementation on embedded devices

Improved accuracy in gesture and posture recognition

Demonstrated feasibility of multimodal sensor fusion in wearables

Abstract

Combining different sensing modalities with multiple positions helps form a unified perception and understanding of complex situations such as human behavior. Hence, human activity recognition (HAR) benefits from combining redundant and complementary information (Unimodal/Multimodal). Even so, it is not an easy task. It requires a multidisciplinary approach, including expertise in sensor technologies, signal processing, data fusion algorithms, and domain-specific knowledge. This Ph.D. work employs sensing modalities such as inertial, pressure (audio and atmospheric pressure), and textile capacitive sensing for HAR. The scenarios explored are gesture and hand position tracking, facial and head pattern recognition, and body posture and gesture recognition. The selected wearable devices and sensing modalities are fully integrated with machine learning-based algorithms, some of which are implemented in the embedded device, on the edge, and tested in real-time.