Contrastive Learning with Auxiliary User Detection for Identifying Activities

TL;DR

This paper introduces CLAUDIA, a framework that combines contrastive learning with user detection to improve human activity recognition by addressing both contextual and user-specific factors, leading to significant performance gains.

Contribution

The paper proposes a novel UCA-HAR task integrating user identification with context-aware HAR and employs contrastive loss to enhance feature learning, which is a new approach in this domain.

Findings

Performance improvements of 5.8% to 14.1% in Matthew's Correlation Coefficient

Enhancement of 3.0% to 7.2% in Macro F1 score

Effective integration of user and context information in HAR models

Abstract

Human Activity Recognition (HAR) is essential in ubiquitous computing, with far-reaching real-world applications. While recent SOTA HAR research has demonstrated impressive performance, some key aspects remain under-explored. Firstly, HAR can be both highly contextualized and personalized. However, prior work has predominantly focused on being Context-Aware (CA) while largely ignoring the necessity of being User-Aware (UA). We argue that addressing the impact of innate user action-performing differences is equally crucial as considering external contextual environment settings in HAR tasks. Secondly, being user-aware makes the model acknowledge user discrepancies but does not necessarily guarantee mitigation of these discrepancies, i.e., unified predictions under the same activities. There is a need for a methodology that explicitly enforces closer (different user, same activity) representations. To bridge this gap, we introduce CLAUDIA, a novel framework designed to address these issues. Specifically, we expand the contextual scope of the CA-HAR task by integrating User Identification (UI) within the CA-HAR framework, jointly predicting both CA-HAR and UI in a new task called User and Context-Aware HAR (UCA-HAR). This approach enriches personalized and contextual understanding by jointly learning user-invariant and user-specific patterns. Inspired by SOTA designs in the visual domain, we introduce a supervised contrastive loss objective on instance-instance pairs to enhance model efficacy and improve learned feature quality. Evaluation across three real-world CA-HAR datasets reveals substantial performance enhancements, with average improvements ranging from 5.8% to 14.1% in Matthew's Correlation Coefficient and 3.0% to 7.2% in Macro F1 score.