PALMAR: Towards Adaptive Multi-inhabitant Activity Recognition in Point-Cloud Technology

TL;DR

PALMAR is an innovative system that combines advanced point-cloud data processing, clustering, and deep learning to accurately recognize activities of multiple inhabitants in real-time, even with limited data and device diversity.

Contribution

The paper introduces novel adaptive techniques for multi-inhabitant activity recognition using point-cloud data, including real-time fine-tuning, multi-person tracking, and domain adaptation methods.

Findings

Achieved 96% activity recognition accuracy in multi-inhabitant scenarios.

Improved multi-person tracking by 63% over existing methods.

Validated system performance on real-time data from multiple devices.

Abstract

With the advancement of deep neural networks and computer vision-based Human Activity Recognition, employment of Point-Cloud Data technologies (LiDAR, mmWave) has seen a lot interests due to its privacy preserving nature. Given the high promise of accurate PCD technologies, we develop, PALMAR, a multiple-inhabitant activity recognition system by employing efficient signal processing and novel machine learning techniques to track individual person towards developing an adaptive multi-inhabitant tracking and HAR system. More specifically, we propose (i) a voxelized feature representation-based real-time PCD fine-tuning method, (ii) efficient clustering (DBSCAN and BIRCH), Adaptive Order Hidden Markov Model based multi-person tracking and crossover ambiguity reduction techniques and (iii) novel adaptive deep learning-based domain adaptation technique to improve the accuracy of HAR in presence of data scarcity and diversity (device, location and population diversity). We experimentally evaluate our framework and systems using (i) a real-time PCD collected by three devices (3D LiDAR and 79 GHz mmWave) from 6 participants, (ii) one publicly available 3D LiDAR activity data (28 participants) and (iii) an embedded hardware prototype system which provided promising HAR performances in multi-inhabitants (96%) scenario with a 63% improvement of multi-person tracking than state-of-art framework without losing significant system performances in the edge computing device.