ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion

TL;DR

This paper introduces ProbRadarM3F, a novel mmWave radar-based human pose estimation model that fuses traditional FFT features with probability map encoding, significantly improving keypoint detection accuracy.

Contribution

The paper presents a new radar feature extraction framework combining FFT and probability map encoding for enhanced pose estimation accuracy.

Findings

Achieved 69.9% AP on HuPR dataset.

Outperformed existing methods in radar-based pose estimation.

Effectively fused traditional and positional features for better keypoint detection.

Abstract

Millimeter wave (mmWave) radar is a non-intrusive privacy and relatively convenient and inexpensive device, which has been demonstrated to be applicable in place of RGB cameras in human indoor pose estimation tasks. However, mmWave radar relies on the collection of reflected signals from the target, and the radar signals containing information is difficult to be fully applied. This has been a long-standing hindrance to the improvement of pose estimation accuracy. To address this major challenge, this paper introduces a probability map guided multi-format feature fusion model, ProbRadarM3F. This is a novel radar feature extraction framework using a traditional FFT method in parallel with a probability map based positional encoding method. ProbRadarM3F fuses the traditional heatmap features and the positional features, then effectively achieves the estimation of 14 keypoints of the human body. Experimental evaluation on the HuPR dataset proves the effectiveness of the model proposed in this paper, outperforming other methods experimented on this dataset with an AP of 69.9 %. The emphasis of our study is focusing on the position information that is not exploited before in radar singal. This provides direction to investigate other potential non-redundant information from mmWave rader.