VCHAR:Variance-Driven Complex Human Activity Recognition framework with Generative Representation

TL;DR

VCHAR is a novel framework that improves complex human activity recognition by modeling atomic activities as distributions and providing interpretable video explanations, reducing labeling effort and enhancing user understanding.

Contribution

It introduces a variance-driven, generative approach that does not require precise atomic activity labels and offers explainability for complex activity recognition.

Findings

Enhanced accuracy on three datasets

More intelligible explanations for non-experts

Reduced need for detailed activity labeling

Abstract

Complex human activity recognition (CHAR) remains a pivotal challenge within ubiquitous computing, especially in the context of smart environments. Existing studies typically require meticulous labeling of both atomic and complex activities, a task that is labor-intensive and prone to errors due to the scarcity and inaccuracies of available datasets. Most prior research has focused on datasets that either precisely label atomic activities or, at minimum, their sequence approaches that are often impractical in real world settings.In response, we introduce VCHAR (Variance-Driven Complex Human Activity Recognition), a novel framework that treats the outputs of atomic activities as a distribution over specified intervals. Leveraging generative methodologies, VCHAR elucidates the reasoning behind complex activity classifications through video-based explanations, accessible to users without prior machine learning expertise. Our evaluation across three publicly available datasets demonstrates that VCHAR enhances the accuracy of complex activity recognition without necessitating precise temporal or sequential labeling of atomic activities. Furthermore, user studies confirm that VCHAR's explanations are more intelligible compared to existing methods, facilitating a broader understanding of complex activity recognition among non-experts.