Human Activity Recognition using Deep Learning Models on Smartphones and Smartwatches Sensor Data

TL;DR

This study compares sensor data from smartphones and smartwatches for human activity recognition, demonstrating that different device placements affect data and model performance, with CNN and ConvLSTM models excelling on smartwatch data.

Contribution

It shows the impact of device placement on sensor data and activity classification, and evaluates multiple neural network architectures for improved recognition accuracy.

Findings

Smartwatches provide better activity classification accuracy than smartphones.

Convolutional neural networks outperform LSTM models in most activities.

Sensor data from different devices are statistically significantly different.

Abstract

In recent years, human activity recognition has garnered considerable attention both in industrial and academic research because of the wide deployment of sensors, such as accelerometers and gyroscopes, in products such as smartphones and smartwatches. Activity recognition is currently applied in various fields where valuable information about an individual's functional ability and lifestyle is needed. In this study, we used the popular WISDM dataset for activity recognition. Using multivariate analysis of covariance (MANCOVA), we established a statistically significant difference (p<0.05) between the data generated from the sensors embedded in smartphones and smartwatches. By doing this, we show that smartphones and smartwatches don't capture data in the same way due to the location where they are worn. We deployed several neural network architectures to classify 15 different hand and non-hand-oriented activities. These models include Long short-term memory (LSTM), Bi-directional Long short-term memory (BiLSTM), Convolutional Neural Network (CNN), and Convolutional LSTM (ConvLSTM). The developed models performed best with watch accelerometer data. Also, we saw that the classification precision obtained with the convolutional input classifiers (CNN and ConvLSTM) was higher than the end-to-end LSTM classifier in 12 of the 15 activities. Additionally, the CNN model for the watch accelerometer was better able to classify non-hand oriented activities when compared to hand-oriented activities.