SoK: Behind the Accuracy of Complex Human Activity Recognition Using Deep Learning

TL;DR

This paper systematically reviews the evolution, challenges, and factors affecting the accuracy of deep learning-based complex human activity recognition, emphasizing sensor modalities and research directions.

Contribution

It provides a comprehensive systematisation of factors influencing HAR accuracy, highlighting challenges and future research directions in deep learning-based complex activity recognition.

Findings

Data variety and model capacity impact accuracy

Wearable and camera sensors are most prevalent

Challenges hinder progress in complex HAR

Abstract

Human Activity Recognition (HAR) is a well-studied field with research dating back to the 1980s. Over time, HAR technologies have evolved significantly from manual feature extraction, rule-based algorithms, and simple machine learning models to powerful deep learning models, from one sensor type to a diverse array of sensing modalities. The scope has also expanded from recognising a limited set of activities to encompassing a larger variety of both simple and complex activities. However, there still exist many challenges that hinder advancement in complex activity recognition using modern deep learning methods. In this paper, we comprehensively systematise factors leading to inaccuracy in complex HAR, such as data variety and model capacity. Among many sensor types, we give more attention to wearable and camera due to their prevalence. Through this Systematisation of Knowledge (SoK) paper, readers can gain a solid understanding of the development history and existing challenges of HAR, different categorisations of activities, obstacles in deep learning-based complex HAR that impact accuracy, and potential research directions.