Neuro-Symbolic Approaches for Context-Aware Human Activity Recognition

TL;DR

This paper introduces a semantic loss function for neuro-symbolic human activity recognition that improves accuracy and deployability on resource-constrained devices without requiring symbolic reasoning during inference.

Contribution

It proposes a novel semantic loss approach that embeds knowledge constraints during training, eliminating the need for symbolic reasoners at runtime.

Findings

Outperforms purely data-driven models on in-the-wild datasets

Achieves recognition rates comparable or better than existing NeSy methods

Enables deployment on resource-limited devices as a single DNN

Abstract

Deep Learning models are a standard solution for sensor-based Human Activity Recognition (HAR), but their deployment is often limited by labeled data scarcity and models' opacity. Neuro-Symbolic AI (NeSy) provides an interesting research direction to mitigate these issues by infusing knowledge about context information into HAR deep learning classifiers. However, existing NeSy methods for context-aware HAR require computationally expensive symbolic reasoners during classification, making them less suitable for deployment on resource-constrained devices (e.g., mobile devices). Additionally, NeSy approaches for context-aware HAR have never been evaluated on in-the-wild datasets, and their generalization capabilities in real-world scenarios are questionable. In this work, we propose a novel approach based on a semantic loss function that infuses knowledge constraints in the HAR model during the training phase, avoiding symbolic reasoning during classification. Our results on scripted and in-the-wild datasets show the impact of different semantic loss functions in outperforming a purely data-driven model. We also compare our solution with existing NeSy methods and analyze each approach's strengths and weaknesses. Our semantic loss remains the only NeSy solution that can be deployed as a single DNN without the need for symbolic reasoning modules, reaching recognition rates close (and better in some cases) to existing approaches.