Heterogeneous Hyper-Graph Neural Networks for Context-aware Human Activity Recognition

TL;DR

This paper introduces a novel heterogeneous hypergraph neural network architecture to improve context-aware human activity recognition by exploiting underlying graph structures in in-the-wild data, significantly outperforming existing methods.

Contribution

The paper proposes a new Heterogeneous HyperGraph Neural Network (HHGNN-CHAR) that models activity recognition data as a hypergraph with multiple node types and hyperedges, enhancing recognition accuracy.

Findings

Significant performance improvements over state-of-the-art baselines.

Effective modeling of <Activity, Phone Placement> as a hypergraph structure.

Robustness demonstrated on in-the-wild datasets.

Abstract

Context-aware Human Activity Recognition (CHAR) is challenging due to the need to recognize the user's current activity from signals that vary significantly with contextual factors such as phone placements and the varied styles with which different users perform the same activity. In this paper, we argue that context-aware activity visit patterns in realistic in-the-wild data can equivocally be considered as a general graph representation learning task. We posit that exploiting underlying graphical patterns in CHAR data can improve CHAR task performance and representation learning. Building on the intuition that certain activities are frequently performed with the phone placed in certain positions, we focus on the context-aware human activity problem of recognizing the <Activity, Phone Placement> tuple. We demonstrate that CHAR data has an underlying graph structure that can be viewed as a heterogenous hypergraph that has multiple types of nodes and hyperedges (an edge connecting more than two nodes). Subsequently, learning <Activity, Phone Placement> representations becomes a graph node representation learning problem. After task transformation, we further propose a novel Heterogeneous HyperGraph Neural Network architecture for Context-aware Human Activity Recognition (HHGNN-CHAR), with three types of heterogeneous nodes (user, phone placement, and activity). Connections between all types of nodes are represented by hyperedges. Rigorous evaluation demonstrated that on an unscripted, in-the-wild CHAR dataset, our proposed framework significantly outperforms state-of-the-art (SOTA) baselines including CHAR models that do not exploit graphs, and GNN variants that do not incorporate heterogeneous nodes or hyperedges with overall improvements 14.04% on Matthews Correlation Coefficient (MCC) and 7.01% on Macro F1 scores.