3D Human Mesh Construction Leveraging Wi-Fi

TL;DR

Wi-Mesh is a novel WiFi-based system that constructs 3D human meshes by leveraging WiFi signals and deep learning, functioning effectively in challenging indoor environments without specialized hardware.

Contribution

The paper introduces Wi-Mesh, a system that uses existing WiFi devices and deep learning to create 3D human models, enabling non-line-of-sight and lighting-independent human shape estimation.

Findings

Achieves 2.81cm vertex location error

Achieves 2.4cm joint position error

Operates effectively in NLoS and poor lighting conditions

Abstract

In this paper, we present, Wi-Mesh, a WiFi vision-based 3D human mesh construction system. Our system leverages the advances of WiFi to visualize the shape and deformations of the human body for 3D mesh construction. In particular, it leverages multiple transmitting and receiving antennas on WiFi devices to estimate the two-dimensional angle of arrival (2D AoA) of the WiFi signal reflections to enable WiFi devices to see the physical environment as we humans do. It then extracts only the images of the human body from the physical environment and leverages deep learning models to digitize the extracted human body into a 3D mesh representation. Experimental evaluation under various indoor environments shows that Wi-Mesh achieves an average vertices location error of 2.81cm and joint position error of 2.4cm, which is comparable to the systems that utilize specialized and dedicated hardware. The proposed system has the advantage of re-using the WiFi devices that already exist in the environment for potential mass adoption. It can also work in non-line of sight (NLoS), poor lighting conditions, and baggy clothes, where the camera-based systems do not work well.