SETransformer: A Hybrid Attention-Based Architecture for Robust Human Activity Recognition

TL;DR

SETransformer is a novel hybrid deep learning architecture that combines Transformer-based temporal modeling with channel-wise attention for improved human activity recognition from wearable sensor data.

Contribution

It introduces a hybrid model integrating Transformer and squeeze-and-excitation attention, enhancing long-range dependency capture and sensor channel emphasis in HAR tasks.

Findings

Outperforms CNN, RNN, LSTM baselines on WISDM dataset

Achieves 84.68% validation accuracy and 84.64% macro F1-score

Demonstrates strong potential for real-world mobile sensing applications

Abstract

Human Activity Recognition (HAR) using wearable sensor data has become a central task in mobile computing, healthcare, and human-computer interaction. Despite the success of traditional deep learning models such as CNNs and RNNs, they often struggle to capture long-range temporal dependencies and contextual relevance across multiple sensor channels. To address these limitations, we propose SETransformer, a hybrid deep neural architecture that combines Transformer-based temporal modeling with channel-wise squeeze-and-excitation (SE) attention and a learnable temporal attention pooling mechanism. The model takes raw triaxial accelerometer data as input and leverages global self-attention to capture activity-specific motion dynamics over extended time windows, while adaptively emphasizing informative sensor channels and critical time steps. We evaluate SETransformer on the WISDM dataset and demonstrate that it significantly outperforms conventional models including LSTM, GRU, BiLSTM, and CNN baselines. The proposed model achieves a validation accuracy of 84.68\% and a macro F1-score of 84.64\%, surpassing all baseline architectures by a notable margin. Our results show that SETransformer is a competitive and interpretable solution for real-world HAR tasks, with strong potential for deployment in mobile and ubiquitous sensing applications.