A benchmark of data stream classification for human activity recognition on connected objects

TL;DR

This study benchmarks data stream classifiers for human activity recognition on connected devices, evaluating their accuracy and resource use, and highlights challenges like high memory needs and low F1 scores.

Contribution

It provides a comprehensive comparison of five stream classifiers for HAR on connected objects, focusing on performance and resource consumption, and identifies key challenges for future research.

Findings

HT, MF, and NB classifiers outperform others on most datasets.

HT and MCNN can recover from concept drift.

All classifiers have lower performance than offline classifiers on real data.

Abstract

This paper evaluates data stream classifiers from the perspective of connected devices, focusing on the use case of HAR. We measure both classification performance and resource consumption (runtime, memory, and power) of five usual stream classification algorithms, implemented in a consistent library, and applied to two real human activity datasets and to three synthetic datasets. Regarding classification performance, results show an overall superiority of the HT, the MF, and the NB classifiers over the FNN and the Micro Cluster Nearest Neighbor (MCNN) classifiers on 4 datasets out of 6, including the real ones. In addition, the HT, and to some extent MCNN, are the only classifiers that can recover from a concept drift. Overall, the three leading classifiers still perform substantially lower than an offline classifier on the real datasets. Regarding resource consumption, the HT and the MF are the most memory intensive and have the longest runtime, however, no difference in power consumption is found between classifiers. We conclude that stream learning for HAR on connected objects is challenged by two factors which could lead to interesting future work: a high memory consumption and low F1 scores overall.