Vehicle Cameras Guide mmWave Beams: Approach and Real-World V2V Demonstration

TL;DR

This paper presents a deep learning approach using 360 camera images to predict mmWave beams in vehicle-to-vehicle communication, significantly improving alignment accuracy and reducing training overhead in real-world scenarios.

Contribution

It introduces a novel vision-based deep learning method for beam prediction in V2V communication, validated on real-world multi-modal datasets.

Findings

Achieves approximately 85% top-5 beam prediction accuracy.

Reduces beam training overhead significantly.

Demonstrates effectiveness in real-world V2V scenarios.

Abstract

Accurately aligning millimeter-wave (mmWave) and terahertz (THz) narrow beams is essential to satisfy reliability and high data rates of 5G and beyond wireless communication systems. However, achieving this objective is difficult, especially in vehicle-to-vehicle (V2V) communication scenarios, where both transmitter and receiver are constantly mobile. Recently, additional sensing modalities, such as visual sensors, have attracted significant interest due to their capability to provide accurate information about the wireless environment. To that end, in this paper, we develop a deep learning solution for V2V scenarios to predict future beams using images from a 360 camera attached to the vehicle. The developed solution is evaluated on a real-world multi-modal mmWave V2V communication dataset comprising co-existing 360 camera and mmWave beam training data. The proposed vision-aided solution achieves $\approx 85\%$ top-5 beam prediction accuracy while significantly reducing the beam training overhead. This highlights the potential of utilizing vision for enabling highly-mobile V2V communications.