MetaFi: Device-Free Pose Estimation via Commodity WiFi for Metaverse Avatar Simulation

TL;DR

MetaFi is a WiFi-based human pose estimation system that enables privacy-preserving, accurate avatar simulation in the metaverse, overcoming limitations of camera-based methods in smart home environments.

Contribution

The paper introduces a novel deep learning approach using WiFi signals for human pose estimation, with cross-modal supervision for improved accuracy in metaverse applications.

Findings

Achieves a PCK@50 of 95.23% in real-world tests.

Provides a privacy-preserving alternative to camera-based pose estimation.

Demonstrates robustness to varying illumination conditions.

Abstract

Avatar refers to a representative of a physical user in the virtual world that can engage in different activities and interact with other objects in metaverse. Simulating the avatar requires accurate human pose estimation. Though camera-based solutions yield remarkable performance, they encounter the privacy issue and degraded performance caused by varying illumination, especially in smart home. In this paper, we propose a WiFi-based IoT-enabled human pose estimation scheme for metaverse avatar simulation, namely MetaFi. Specifically, a deep neural network is designed with customized convolutional layers and residual blocks to map the channel state information to human pose landmarks. It is enforced to learn the annotations from the accurate computer vision model, thus achieving cross-modal supervision. WiFi is ubiquitous and robust to illumination, making it a feasible solution for avatar applications in smart home. The experiments are conducted in the real world, and the results show that the MetaFi achieves very high performance with a PCK@50 of 95.23%.