USAD: End-to-End Human Activity Recognition via Diffusion Model with Spatiotemporal Attention

TL;DR

This paper introduces USAD, a novel end-to-end human activity recognition model combining diffusion-based data augmentation with multi-scale spatio-temporal attention mechanisms, achieving high accuracy and efficiency on lightweight devices.

Contribution

The paper presents a comprehensive approach integrating an unsupervised diffusion model for data augmentation with a multi-branch attention network for improved feature extraction in HAR.

Findings

Achieved over 98% accuracy on WISDM dataset.

Outperformed existing methods on PAMAP2 and OPPORTUNITY datasets.

Validated efficiency on embedded devices.

Abstract

The primary objective of human activity recognition (HAR) is to infer ongoing human actions from sensor data, a task that finds broad applications in health monitoring, safety protection, and sports analysis. Despite proliferating research, HAR still faces key challenges, including the scarcity of labeled samples for rare activities, insufficient extraction of high-level features, and suboptimal model performance on lightweight devices. To address these issues, this paper proposes a comprehensive optimization approach centered on multi-attention interaction mechanisms. First, an unsupervised, statistics-guided diffusion model is employed to perform data augmentation, thereby alleviating the problems of labeled data scarcity and severe class imbalance. Second, a multi-branch spatio-temporal interaction network is designed, which captures multi-scale features of sequential data through parallel residual branches with 3*3, 5*5, and 7*7 convolutional kernels. Simultaneously, temporal attention mechanisms are incorporated to identify critical time points, while spatial attention enhances inter-sensor interactions. A cross-branch feature fusion unit is further introduced to improve the overall feature representation capability. Finally, an adaptive multi-loss function fusion strategy is integrated, allowing for dynamic adjustment of loss weights and overall model optimization. Experimental results on three public datasets, WISDM, PAMAP2, and OPPORTUNITY, demonstrate that the proposed unsupervised data augmentation spatio-temporal attention diffusion network (USAD) achieves accuracies of 98.84%, 93.81%, and 80.92% respectively, significantly outperforming existing approaches. Furthermore, practical deployment on embedded devices verifies the efficiency and feasibility of the proposed method.