Self-supervised One-Stage Learning for RF-based Multi-Person Pose Estimation

TL;DR

This paper introduces a lightweight, one-stage RF-based multi-person pose estimation model utilizing self-supervised learning, significantly improving accuracy and robustness in challenging conditions over previous methods.

Contribution

It proposes a novel efficient RF-based MPPE model with a self-supervised learning approach that enhances generalization and accuracy, especially in new locations and obstacle scenarios.

Findings

Improves MPPE accuracy by up to 15 in PCKh@0.5 over previous RF methods.

Self-supervised learning significantly boosts performance in new environments.

Model performs well with increasing number of people and obstructions.

Abstract

In the field of Multi-Person Pose Estimation (MPPE), Radio Frequency (RF)-based methods can operate effectively regardless of lighting conditions and obscured line-of-sight situations. Existing RF-based MPPE methods typically involve either 1) converting RF signals into heatmap images through complex preprocessing, or 2) applying a deep embedding network directly to raw RF signals. The first approach, while delivering decent performance, is computationally intensive and time-consuming. The second method, though simpler in preprocessing, results in lower MPPE accuracy and generalization performance. This paper proposes an efficient and lightweight one-stage MPPE model based on raw RF signals. By sub-grouping RF signals and embedding them using a shared single-layer CNN followed by multi-head attention, this model outperforms previous methods that embed all signals at once through a large and deep CNN. Additionally, we propose a new self-supervised learning (SSL) method that takes inputs from both one unmasked subgroup and the remaining masked subgroups to predict the latent representations of the masked data. Empirical results demonstrate that our model improves MPPE accuracy by up to 15 in PCKh@0.5 compared to previous methods using raw RF signals. Especially, the proposed SSL method has shown to significantly enhance performance improvements when placed in new locations or in front of obstacles at RF antennas, contributing to greater performance gains as the number of people increases. Our code and dataset is open at Github. https://github.com/sshnan7/SOSPE .