Enabling Visual Recognition at Radio Frequency

TL;DR

PanoRadar is a novel RF imaging system that achieves LiDAR-like 3D imaging resolution at radio frequencies, enabling visual recognition tasks such as object detection and segmentation in challenging conditions.

Contribution

The paper presents PanoRadar, a new RF imaging system combining advanced signal processing and machine learning to produce high-resolution 3D images comparable to LiDAR.

Findings

Achieves LiDAR-like 3D imaging resolution at RF frequencies

Enables visual recognition tasks like segmentation and object detection at radio frequencies

Demonstrates robust performance across multiple buildings

Abstract

This paper introduces PanoRadar, a novel RF imaging system that brings RF resolution close to that of LiDAR, while providing resilience against conditions challenging for optical signals. Our LiDAR-comparable 3D imaging results enable, for the first time, a variety of visual recognition tasks at radio frequency, including surface normal estimation, semantic segmentation, and object detection. PanoRadar utilizes a rotating single-chip mmWave radar, along with a combination of novel signal processing and machine learning algorithms, to create high-resolution 3D images of the surroundings. Our system accurately estimates robot motion, allowing for coherent imaging through a dense grid of synthetic antennas. It also exploits the high azimuth resolution to enhance elevation resolution using learning-based methods. Furthermore, PanoRadar tackles 3D learning via 2D convolutions and addresses challenges due to the unique characteristics of RF signals. Our results demonstrate PanoRadar's robust performance across 12 buildings.