Wave-Former: Through-Occlusion 3D Reconstruction via Wireless Shape Completion

TL;DR

Wave-Former is a new method that uses millimeter-wave wireless signals and a three-stage pipeline to accurately reconstruct 3D shapes of occluded objects, enabling advancements in robotics, AR, and logistics.

Contribution

It introduces a physics-aware shape completion model that bridges wireless signals with vision-based methods, allowing high-accuracy 3D reconstruction from synthetic training data.

Findings

Increased recall from 54% to 72% over baselines

Achieved 85% precision in 3D shape reconstruction

Demonstrated effective generalization to real-world data

Abstract

We present Wave-Former, a novel method capable of high-accuracy 3D shape reconstruction for completely occluded, diverse, everyday objects. This capability can open new applications spanning robotics, augmented reality, and logistics. Our approach leverages millimeter-wave (mmWave) wireless signals, which can penetrate common occlusions and reflect off hidden objects. In contrast to past mmWave reconstruction methods, which suffer from limited coverage and high noise, Wave-Former introduces a physics-aware shape completion model capable of inferring full 3D geometry. At the heart of Wave-Former's design is a novel three-stage pipeline which bridges raw wireless signals with recent advancements in vision-based shape completion by incorporating physical properties of mmWave signals. The pipeline proposes candidate geometric surfaces, employs a transformer-based shape completion model designed specifically for mmWave signals, and finally performs entropy-guided surface selection. This enables Wave-Former to be trained using entirely synthetic point-clouds, while demonstrating impressive generalization to real-world data. In head-to-head comparisons with state-of-the-art baselines, Wave-Former raises recall from 54% to 72% while maintaining a high precision of 85%.