Transformer-based Models to Deal with Heterogeneous Environments in Human Activity Recognition

TL;DR

This paper introduces Transformer-based models, HART and MobileHART, for human activity recognition that are more robust to data heterogeneity and device variability in real-world scenarios, outperforming previous models with fewer resources.

Contribution

The paper proposes two novel sensor-wise Transformer architectures, HART and MobileHART, specifically designed to handle data heterogeneity in human activity recognition tasks.

Findings

HART and MobileHART outperform previous architectures in accuracy.

They require fewer floating point operations and parameters.

They are more robust to device and position changes.

Abstract

Human Activity Recognition (HAR) on mobile devices has been demonstrated to be possible using neural models trained on data collected from the device's inertial measurement units. These models have used Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTMs), Transformers or a combination of these to achieve state-of-the-art results with real-time performance. However, these approaches have not been extensively evaluated in real-world situations where the input data may be different from the training data. This paper highlights the issue of data heterogeneity in machine learning applications and how it can hinder their deployment in pervasive settings. To address this problem, we propose and publicly release the code of two sensor-wise Transformer architectures called HART and MobileHART for Human Activity Recognition Transformer. Our experiments on several publicly available datasets show that these HART architectures outperform previous architectures with fewer floating point operations and parameters than conventional Transformers. The results also show they are more robust to changes in mobile position or device brand and hence better suited for the heterogeneous environments encountered in real-life settings. Finally, the source code has been made publicly available.