A Tree-structure Convolutional Neural Network for Temporal Features Exaction on Sensor-based Multi-resident Activity Recognition

TL;DR

This paper introduces an end-to-end tree-structure convolutional neural network that automatically learns temporal features from sensor data for multi-resident activity recognition in smart homes, overcoming manual segmentation limitations.

Contribution

The proposed TSC-MRAR framework uniquely models temporal dependencies with a tree-structure CNN and jointly predicts resident and activity labels without manual data segmentation.

Findings

Outperforms state-of-the-art methods on CASAS datasets

Effectively captures temporal dependencies in sensor data

Jointly recognizes multiple residents and activities

Abstract

With the propagation of sensor devices applied in smart home, activity recognition has ignited huge interest and most existing works assume that there is only one habitant. While in reality, there are generally multiple residents at home, which brings greater challenge to recognize activities. In addition, many conventional approaches rely on manual time series data segmentation ignoring the inherent characteristics of events and their heuristic hand-crafted feature generation algorithms are difficult to exploit distinctive features to accurately classify different activities. To address these issues, we propose an end-to-end Tree-Structure Convolutional neural network based framework for Multi-Resident Activity Recognition (TSC-MRAR). First, we treat each sample as an event and obtain the current event embedding through the previous sensor readings in the sliding window without splitting the time series data. Then, in order to automatically generate the temporal features, a tree-structure network is designed to derive the temporal dependence of nearby readings. The extracted features are fed into the fully connected layer, which can jointly learn the residents labels and the activity labels simultaneously. Finally, experiments on CASAS datasets demonstrate the high performance in multi-resident activity recognition of our model compared to state-of-the-art techniques.