"Filling the Blanks'': Identifying Micro-activities that Compose Complex Human Activities of Daily Living

TL;DR

This paper introduces AmicroN, a novel top-down method that uses coarse annotations and unsupervised change-point detection to identify micro-activities within complex daily activities, improving interpretability without requiring detailed labels.

Contribution

AmicroN is the first approach to expand macro-activities into micro-activities using only coarse labels and unsupervised techniques, eliminating the need for fine-grained annotations.

Findings

Achieves micro F1-score > 0.75 on two real datasets.

Effectively identifies micro-activities without detailed supervision.

Demonstrates potential for enhanced explainability with large language models.

Abstract

Complex activities of daily living (ADLs) often consist of multiple micro-activities. When performed sequentially, these micro-activities help the user accomplish the broad macro-activity. Naturally, a deeper understanding of these micro-activities can help develop more sophisticated human activity recognition (HAR) models and add explainability to their inferred conclusions. Previous research has attempted to achieve this by utilizing fine-grained annotated data that provided the required supervision and rules for associating the micro-activities to identify the macro-activity. However, this ``bottom-up'' approach is unrealistic as getting such high-quality, fine-grained annotated sensor datasets is challenging, costly, and time-consuming. Understanding this, in this paper, we develop AmicroN, which adapts a ``top-down'' approach by exploiting coarse-grained annotated data to expand the macro-activities into their constituent micro-activities without any external supervision. In the backend, AmicroN uses \textit{unsupervised} change-point detection to search for the micro-activity boundaries across a complex ADL. Then, it applies a \textit{generalized zero-shot} approach to characterize it. We evaluate AmicroN on two real-life publicly available datasets and observe that AmicroN can identify the micro-activities with micro F\textsubscript{1}-score $>0.75$ for both datasets. Additionally, we also perform an initial proof-of-concept on leveraging the state-of-the-art (SOTA) large language models (LLMs) with attribute embeddings predicted by AmicroN to enhance further the explainability surrounding the detection of micro-activities.