Vision-Position Multi-Modal Beam Prediction Using Real Millimeter Wave Datasets

TL;DR

This paper introduces a multi-modal machine learning framework that uses positional and visual data to predict mmWave beam directions, significantly reducing training overhead in high-mobility wireless systems.

Contribution

It presents a novel approach combining visual and positional data for fast beam prediction in mmWave systems, validated on real-world vehicular datasets.

Findings

Achieves over 75% top-1 beam prediction accuracy.

Reaches nearly 100% top-3 beam prediction accuracy.

Effective in realistic vehicular communication scenarios.

Abstract

Enabling highly-mobile millimeter wave (mmWave) and terahertz (THz) wireless communication applications requires overcoming the critical challenges associated with the large antenna arrays deployed at these systems. In particular, adjusting the narrow beams of these antenna arrays typically incurs high beam training overhead that scales with the number of antennas. To address these challenges, this paper proposes a multi-modal machine learning based approach that leverages positional and visual (camera) data collected from the wireless communication environment for fast beam prediction. The developed framework has been tested on a real-world vehicular dataset comprising practical GPS, camera, and mmWave beam training data. The results show the proposed approach achieves more than $\approx$ 75\% top-1 beam prediction accuracy and close to 100\% top-3 beam prediction accuracy in realistic communication scenarios.