FLOW: Fusing and Shuffling Global and Local Views for Cross-User Human Activity Recognition with IMUs

TL;DR

This paper introduces a novel multi-view fusion approach for cross-user human activity recognition using IMU sensors, leveraging global and local data views to improve generalization across diverse users.

Contribution

It proposes a global view representation to mitigate user variation effects and a shuffling-based fusion network (MVFNet) to enhance feature integration for cross-user HAR.

Findings

Global view data outperforms local view in cross-user tests.

The proposed method surpasses state-of-the-art in OPPORTUNITY and PAMAP2 datasets.

Extensive experiments validate the effectiveness of the fusion approach.

Abstract

Inertial Measurement Unit (IMU) sensors are widely employed for Human Activity Recognition (HAR) due to their portability, energy efficiency, and growing research interest. However, a significant challenge for IMU-HAR models is achieving robust generalization performance across diverse users. This limitation stems from substantial variations in data distribution among individual users. One primary reason for this distribution disparity lies in the representation of IMU sensor data in the local coordinate system, which is susceptible to subtle user variations during IMU wearing. To address this issue, we propose a novel approach that extracts a global view representation based on the characteristics of IMU data, effectively alleviating the data distribution discrepancies induced by wearing styles. To validate the efficacy of the global view representation, we fed both global and local view data into model for experiments. The results demonstrate that global view data significantly outperforms local view data in cross-user experiments. Furthermore, we propose a Multi-view Supervised Network (MVFNet) based on Shuffling to effectively fuse local view and global view data. It supervises the feature extraction of each view through view division and view shuffling, so as to avoid the model ignoring important features as much as possible. Extensive experiments conducted on OPPORTUNITY and PAMAP2 datasets demonstrate that the proposed algorithm outperforms the current state-of-the-art methods in cross-user HAR.