mmWave Radar-Based Non-Line-of-Sight Pedestrian Localization at T-Junctions Utilizing Road Layout Extraction via Camera

TL;DR

This paper introduces a novel framework combining mmWave radar and camera data to accurately localize pedestrians in NLoS regions at T-junctions, addressing challenges of multipath distortion and lack of depth perception.

Contribution

It presents a new method that interprets radar point clouds using camera-derived road layouts for improved NLoS pedestrian localization.

Findings

Effective localization in outdoor NLoS environments demonstrated

Radar-camera system achieves accurate spatial scene reconstruction

Method outperforms existing approaches in complex urban scenarios

Abstract

Pedestrians Localization in Non-Line-of-Sight (NLoS) regions within urban environments poses a significant challenge for autonomous driving systems. While mmWave radar has demonstrated potential for detecting objects in such scenarios, the 2D radar point cloud (PCD) data is susceptible to distortions caused by multipath reflections, making accurate spatial inference difficult. Additionally, although camera images provide high-resolution visual information, they lack depth perception and cannot directly observe objects in NLoS regions. In this paper, we propose a novel framework that interprets radar PCD through road layout inferred from camera for localization of NLoS pedestrians. The proposed method leverages visual information from the camera to interpret 2D radar PCD, enabling spatial scene reconstruction. The effectiveness of the proposed approach is validated through experiments conducted using a radar-camera system mounted on a real vehicle. The localization performance is evaluated using a dataset collected in outdoor NLoS driving environments, demonstrating the practical applicability of the method.