Learning to Detect Open Carry and Concealed Object with 77GHz Radar

TL;DR

This paper presents a real-time system using 77GHz mmWave radar and deep learning to detect open carry and concealed objects like laptops, phones, and knives, with extensive validation and analysis.

Contribution

First baseline system for detecting carried objects with 77GHz radar, combining signal processing, deep learning, and multi-shot post-processing.

Findings

Effective detection of open carry and concealed objects

System performance influenced by input formats and parameters

Provides a foundation for future radar-based object detection research

Abstract

Detecting harmful carried objects plays a key role in intelligent surveillance systems and has widespread applications, for example, in airport security. In this paper, we focus on the relatively unexplored area of using low-cost 77GHz mmWave radar for the carried objects detection problem. The proposed system is capable of real-time detecting three classes of objects - laptop, phone, and knife - under open carry and concealed cases where objects are hidden with clothes or bags. This capability is achieved by the initial signal processing for localization and generating range-azimuth-elevation image cubes, followed by a deep learning-based prediction network and a multi-shot post-processing module for detecting objects. Extensive experiments for validating the system performance on detecting open carry and concealed objects have been presented with a self-built radar-camera testbed and collected dataset. Additionally, the influence of different input formats, factors, and parameters on system performance is analyzed, providing an intuitive understanding of the system. This system would be the very first baseline for other future works aiming to detect carried objects using 77GHz radar.