A Two-Stage Motion-Aware Framework for mmWave-based Human Mesh Recovery

TL;DR

This paper presents a two-stage radar-based human mesh recovery framework that enhances accuracy by decoupling signal interpretation from geometric reasoning and leveraging motion cues.

Contribution

It introduces a novel two-stage approach with a reflection extraction module and a motion-aware mesh recovery network for improved radar-based human reconstruction.

Findings

Outperforms existing methods in accuracy

Maintains computational efficiency

Effectively models inter-frame dynamics

Abstract

Millimeter-wave (mmWave) radar has emerged as a promising sensing modality for human perception due to its robustness under challenging environmental conditions and strong privacy-preserving properties. However, recovering accurate 3D human body meshes from radar observations remains difficult due to severe signal clutter and the inherently partial nature of radar measurements. Previous works typically adopt end-to-end frameworks that directly regress human body parameters from raw radar data, without decoupling signal interpretation from geometric reasoning or exploiting temporal motion cues, limiting learning performance. To address this, we propose a two-stage framework for radar-based human body reconstruction. First, we introduce a human reflection extraction module that performs coarse-to-fine localization and voxel-wise segmentation to produce a confidence-weighted radar volume encoding voxel-level human likelihood. Second, we design a motion-aware mesh recovery network that reconstructs the human body by jointly modeling per-frame geometry and inter-frame dynamics using a dual-branch architecture. Extensive experiments demonstrate that the proposed method outperforms existing approaches while maintaining computational efficiency.