3D Reconstruction of Multiple Objects by mmWave Radar on UAV

TL;DR

This paper investigates using a mmWave radar on a UAV to reconstruct 3D shapes of multiple objects, demonstrating the feasibility and robustness of deep learning models in noisy, real-world conditions.

Contribution

It introduces and evaluates two deep neural network models for 3D object reconstruction from UAV-mounted radar data, including a segmentation-enhanced variant.

Findings

Both models successfully reconstruct multiple objects.

Model 2 produces denser, smoother point clouds but may increase reconstruction loss.

Reconstruction is robust to noisy radar data from UAV vibrations.

Abstract

In this paper, we explore the feasibility of utilizing a mmWave radar sensor installed on a UAV to reconstruct the 3D shapes of multiple objects in a space. The UAV hovers at various locations in the space, and its onboard radar senor collects raw radar data via scanning the space with Synthetic Aperture Radar (SAR) operation. The radar data is sent to a deep neural network model, which outputs the point cloud reconstruction of the multiple objects in the space. We evaluate two different models. Model 1 is our recently proposed 3DRIMR/R2P model, and Model 2 is formed by adding a segmentation stage in the processing pipeline of Model 1. Our experiments have demonstrated that both models are promising in solving the multiple object reconstruction problem. We also show that Model 2, despite producing denser and smoother point clouds, can lead to higher reconstruction loss or even loss of objects. In addition, we find that both models are robust to the highly noisy radar data obtained by unstable SAR operation due to the instability or vibration of a small UAV hovering at its intended scanning point. Our exploratory study has shown a promising direction of applying mmWave radar sensing in 3D object reconstruction.