Skeleton Detection Using Dual Radars with Integration of Dual-View CNN Models and mmPose

TL;DR

This paper introduces dual-view CNN models that fuse point cloud data from two mmWave radars with PointNet and mmPose for skeleton detection, aiming to improve elderly fall detection while preserving privacy.

Contribution

It presents a novel dual-radar fusion approach combining PointNet and mmPose for skeleton detection using radar data, addressing data sparsity and invariance issues.

Findings

Model performs well in arm swing detection.

Suboptimal results in random walking scenarios.

Fusion of dual radars enhances skeletal detection accuracy.

Abstract

Skeleton detection is a technique that can beapplied to a variety of situations. It is especially critical identifying and tracking the movements of the elderly, especially in real-time fall detection. While conventional image processing methods exist, there's a growing preference for utilizing pointclouds data collected by mmWave radars from viewpoint of privacy protection, offering a non-intrusive approach to elevatesafety and care for the elderly. Dealing with point cloud data necessitates addressing three critical considerations. Firstly, the inherent nature of point clouds -- rotation invariance, translation invariance, and locality -- is managed through the fusion of PointNet and mmPose. PointNet ensures rotational and translational invariance, while mmPose addresses locality. Secondly, the limited points per frame from radar require data integration from two radars to enhance skeletal detection. Lastly,inputting point cloud data into the learning model involves utilizing features like coordinates, velocity, and signal-to-noise ratio (SNR) per radar point to mitigate sparsity issues and reduce computational load. This research proposes three Dual ViewCNN models, combining PointNet and mmPose, employing two mmWave radars, with performance comparisons in terms of Mean Absolute Error (MAE). While the proposed model shows suboptimal results for random walking, it excels in the arm swing case.