Using Language Model to Bootstrap Human Activity Recognition Ambient Sensors Based in Smart Homes

TL;DR

This paper enhances LSTM-based activity recognition in smart homes by integrating NLP embeddings like Word2Vec and ELMo to incorporate sensor semantics and context, improving classification accuracy and enabling transfer learning.

Contribution

It introduces the use of NLP embedding methods to incorporate sensor semantics into activity recognition models, demonstrating improved performance and transfer learning capabilities.

Findings

Embedding methods reduce confusion between similar activities.

Pretrained embeddings enable transfer learning across datasets.

Sensor semantics improve activity classification accuracy.

Abstract

Long Short Term Memory LSTM-based structures have demonstrated their efficiency for daily living recognition activities in smart homes by capturing the order of sensor activations and their temporal dependencies. Nevertheless, they still fail in dealing with the semantics and the context of the sensors. More than isolated id and their ordered activation values, sensors also carry meaning. Indeed, their nature and type of activation can translate various activities. Their logs are correlated with each other, creating a global context. We propose to use and compare two Natural Language Processing embedding methods to enhance LSTM-based structures in activity-sequences classification tasks: Word2Vec, a static semantic embedding, and ELMo, a contextualized embedding. Results, on real smart homes datasets, indicate that this approach provides useful information, such as a sensor organization map, and makes less confusion between daily activity classes. It helps to better perform on datasets with competing activities of other residents or pets. Our tests show also that the embeddings can be pretrained on different datasets than the target one, enabling transfer learning. We thus demonstrate that taking into account the context of the sensors and their semantics increases the classification performances and enables transfer learning.