mmWEAVER: Environment-Specific mmWave Signal Synthesis from a Photo and Activity Description

TL;DR

mmWeaver is a novel framework that synthesizes realistic, environment-specific mmWave signals using implicit neural representations, enabling efficient dataset augmentation for radar applications with improved accuracy and speed.

Contribution

It introduces a method that models mmWave signals as continuous functions conditioned on environment and activity, achieving high compression and realism, surpassing existing simulation techniques.

Findings

Achieves up to 49-fold data compression.

Outperforms existing methods in signal realism (SSIM 0.88, PSNR 35 dB).

Improves activity recognition accuracy by 7% and pose estimation by 15%.

Abstract

Realistic signal generation and dataset augmentation are essential for advancing mmWave radar applications such as activity recognition and pose estimation, which rely heavily on diverse, and environment-specific signal datasets. However, mmWave signals are inherently complex, sparse, and high-dimensional, making physical simulation computationally expensive. This paper presents mmWeaver, a novel framework that synthesizes realistic, environment-specific complex mmWave signals by modeling them as continuous functions using Implicit Neural Representations (INRs), achieving up to 49-fold compression. mmWeaver incorporates hypernetworks that dynamically generate INR parameters based on environmental context (extracted from RGB-D images) and human motion features (derived from text-to-pose generation via MotionGPT), enabling efficient and adaptive signal synthesis. By conditioning on these semantic and geometric priors, mmWeaver generates diverse I/Q signals at multiple resolutions, preserving phase information critical for downstream tasks such as point cloud estimation and activity classification. Extensive experiments show that mmWeaver achieves a complex SSIM of 0.88 and a PSNR of 35 dB, outperforming existing methods in signal realism while improving activity recognition accuracy by up to 7% and reducing human pose estimation error by up to 15%, all while operating 6-35 times faster than simulation-based approaches.