System-Level Comparison of Multimodal and In-Band mmWave Sensing for Beam Prediction in 6G ISAC

TL;DR

This paper develops a system-level framework to evaluate multimodal sensing methods for beam prediction in 6G V2I links, demonstrating high accuracy with various sensor combinations and establishing baselines for ISAC-assisted beam prediction.

Contribution

The work introduces a comprehensive system-level evaluation framework for multimodal sensing in 6G ISAC, including a latency-aware neural network for beam prediction and performance benchmarks.

Findings

mmWave power vector alone predicts beam index with 98% Top-5 accuracy

Fusing exteroceptive sensors with mmWave maintains high accuracy, e.g., mmWave+LiDAR/GPS/Radar achieves 98% Top-5 accuracy

The framework provides calibrated baselines for 6G ISAC beam prediction in V2I systems

Abstract

Integrated sensing and communication (ISAC) can reduce beam-training overhead in mmWave vehicle-to-infrastructure (V2I) links by enabling in-band sensing-based beam prediction, while exteroceptive sensors can further enhance the prediction accuracy. This work develop a system-level framework that evaluates camera, LiDAR, radar, GPS, and in-band mmWave power, both individually and in multimodal fusion using the DeepSense-6G Scenario-33 dataset. A latency-aware neural network composed of lightweight convolutional (CNN) and multilayer-perceptron (MLP) encoders predict a 64-beam index. We assess performance using Top-k accuracy alongside spectral-efficiency (SE) gap, signal-to-noise-ratio (SNR) gap, rate loss, and end-to-end latency. Results show that the mmWave power vector is a strong standalone predictor, and fusing exteroceptive sensors with it preserves high performance: mmWave alone and mmWave+LiDAR/GPS/Radar achieve 98% Top-5 accuracy, while mmWave+camera achieves 94% Top-5 accuracy. The proposed framework establishes calibrated baselines for 6G ISAC-assisted beam prediction in V2I systems.