Human Activity Recognition Based on Wearable Sensor Data: A Standardization of the State-of-the-Art

TL;DR

This paper critically analyzes the evaluation protocols in wearable sensor-based human activity recognition, demonstrating that current methods often produce skewed results and proposing standardized evaluation practices to improve comparability and reliability.

Contribution

It introduces a comprehensive analysis of sample generation and validation protocols, highlighting their impact on recognition performance and proposing standardization for better assessment.

Findings

Recognition accuracy drops with improper evaluation protocols.

Current evaluation methods are inadequate for wearable sensor data.

Standardized protocols improve reliability of activity recognition results.

Abstract

Human activity recognition based on wearable sensor data has been an attractive research topic due to its application in areas such as healthcare and smart environments. In this context, many works have presented remarkable results using accelerometer, gyroscope and magnetometer data to represent the activities categories. However, current studies do not consider important issues that lead to skewed results, making it hard to assess the quality of sensor-based human activity recognition and preventing a direct comparison of previous works. These issues include the samples generation processes and the validation protocols used. We emphasize that in other research areas, such as image classification and object detection, these issues are already well-defined, which brings more efforts towards the application. Inspired by this, we conduct an extensive set of experiments that analyze different sample generation processes and validation protocols to indicate the vulnerable points in human activity recognition based on wearable sensor data. For this purpose, we implement and evaluate several top-performance methods, ranging from handcrafted-based approaches to convolutional neural networks. According to our study, most of the experimental evaluations that are currently employed are not adequate to perform the activity recognition in the context of wearable sensor data, in which the recognition accuracy drops considerably when compared to an appropriate evaluation approach. To the best of our knowledge, this is the first study that tackles essential issues that compromise the understanding of the performance in human activity recognition based on wearable sensor data.