Beyond Isolated Frames: Enhancing Sensor-Based Human Activity Recognition through Intra- and Inter-Frame Attention

TL;DR

This paper introduces an intra- and inter-frame attention model combined with a novel batch learning strategy to improve sensor-based human activity recognition by capturing both local and global temporal dynamics.

Contribution

It presents a new attention-based model and a time-sequential batch learning method to better capture temporal dependencies in HAR data.

Findings

Enhanced recognition accuracy demonstrated over baseline models

Effective modeling of both intra- and inter-frame relationships

Preservation of temporal sequence improves model performance

Abstract

Human Activity Recognition (HAR) has become increasingly popular with ubiquitous computing, driven by the popularity of wearable sensors in fields like healthcare and sports. While Convolutional Neural Networks (ConvNets) have significantly contributed to HAR, they often adopt a frame-by-frame analysis, concentrating on individual frames and potentially overlooking the broader temporal dynamics inherent in human activities. To address this, we propose the intra- and inter-frame attention model. This model captures both the nuances within individual frames and the broader contextual relationships across multiple frames, offering a comprehensive perspective on sequential data. We further enrich the temporal understanding by proposing a novel time-sequential batch learning strategy. This learning strategy preserves the chronological sequence of time-series data within each batch, ensuring the continuity and integrity of temporal patterns in sensor-based HAR.