UMSNet: An Universal Multi-sensor Network for Human Activity Recognition

TL;DR

UMSNet is a versatile, lightweight neural network that effectively processes multimodal sensor data for human activity recognition, utilizing novel residual blocks and Transformer-based feature relationship modeling.

Contribution

The paper introduces a universal multi-sensor network with a new lightweight residual block and Transformer integration for improved activity recognition.

Findings

Outperforms state-of-the-art methods on HHAR and MHEALTH datasets.

Efficiently processes various multi-modal time series data.

Demonstrates high accuracy and generalization across datasets.

Abstract

Human activity recognition (HAR) based on multimodal sensors has become a rapidly growing branch of biometric recognition and artificial intelligence. However, how to fully mine multimodal time series data and effectively learn accurate behavioral features has always been a hot topic in this field. Practical applications also require a well-generalized framework that can quickly process a variety of raw sensor data and learn better feature representations. This paper proposes a universal multi-sensor network (UMSNet) for human activity recognition. In particular, we propose a new lightweight sensor residual block (called LSR block), which improves the performance by reducing the number of activation function and normalization layers, and adding inverted bottleneck structure and grouping convolution. Then, the Transformer is used to extract the relationship of series features to realize the classification and recognition of human activities. Our framework has a clear structure and can be directly applied to various types of multi-modal Time Series Classification (TSC) tasks after simple specialization. Extensive experiments show that the proposed UMSNet outperforms other state-of-the-art methods on two popular multi-sensor human activity recognition datasets (i.e. HHAR dataset and MHEALTH dataset).