3DRIMR: 3D Reconstruction and Imaging via mmWave Radar based on Deep Learning

TL;DR

This paper introduces 3DRIMR, a deep learning framework using conditional GANs to reconstruct detailed 3D object shapes from sparse mmWave radar data, overcoming challenges like noise and low resolution.

Contribution

The paper presents a novel deep learning architecture combining two GANs to convert raw radar data into detailed 3D point clouds, advancing radar-based 3D reconstruction methods.

Findings

Effective 3D shape reconstruction from sparse radar data

Performance improvement over standard techniques

Demonstrated capability in dense detailed point cloud generation

Abstract

mmWave radar has been shown as an effective sensing technique in low visibility, smoke, dusty, and dense fog environment. However tapping the potential of radar sensing to reconstruct 3D object shapes remains a great challenge, due to the characteristics of radar data such as sparsity, low resolution, specularity, high noise, and multi-path induced shadow reflections and artifacts. In this paper we propose 3D Reconstruction and Imaging via mmWave Radar (3DRIMR), a deep learning based architecture that reconstructs 3D shape of an object in dense detailed point cloud format, based on sparse raw mmWave radar intensity data. The architecture consists of two back-to-back conditional GAN deep neural networks: the first generator network generates 2D depth images based on raw radar intensity data, and the second generator network outputs 3D point clouds based on the results of the first generator. The architecture exploits both convolutional neural network's convolutional operation (that extracts local structure neighborhood information) and the efficiency and detailed geometry capture capability of point clouds (other than costly voxelization of 3D space or distance fields). Our experiments have demonstrated 3DRIMR's effectiveness in reconstructing 3D objects, and its performance improvement over standard techniques.