Hierarchical Self Attention Based Autoencoder for Open-Set Human Activity Recognition

TL;DR

This paper introduces a hierarchical self-attention autoencoder for open-set human activity recognition, improving detection of unseen activities and robustness to noise using explainable feature selection across multiple datasets.

Contribution

It presents a novel hierarchical self-attention autoencoder that enhances open-set activity recognition and model interpretability in wearable sensor data.

Findings

Significant performance improvement over state-of-the-art models.

Enhanced robustness to noise and subject variability.

Explainable attention maps for feature selection.

Abstract

Wearable sensor based human activity recognition is a challenging problem due to difficulty in modeling spatial and temporal dependencies of sensor signals. Recognition models in closed-set assumption are forced to yield members of known activity classes as prediction. However, activity recognition models can encounter an unseen activity due to body-worn sensor malfunction or disability of the subject performing the activities. This problem can be addressed through modeling solution according to the assumption of open-set recognition. Hence, the proposed self attention based approach combines data hierarchically from different sensor placements across time to classify closed-set activities and it obtains notable performance improvement over state-of-the-art models on five publicly available datasets. The decoder in this autoencoder architecture incorporates self-attention based feature representations from encoder to detect unseen activity classes in open-set recognition setting. Furthermore, attention maps generated by the hierarchical model demonstrate explainable selection of features in activity recognition. We conduct extensive leave one subject out validation experiments that indicate significantly improved robustness to noise and subject specific variability in body-worn sensor signals. The source code is available at: github.com/saif-mahmud/hierarchical-attention-HAR