Multi-Modal Large Models Based Beam Prediction: An Example Empowered by DeepSeek

TL;DR

This paper introduces MLM-BP, a multi-modal large model framework for beam prediction in MIMO systems, leveraging environmental data to significantly improve accuracy and generalization, especially in limited data scenarios.

Contribution

The paper presents MLM-BP, a novel multi-modal large model-based framework that integrates environmental information for enhanced beam prediction in MIMO systems.

Findings

Achieves 98.1% Top-1 accuracy in simulation

Demonstrates few-shot generalization with 72.7% Top-1 accuracy on real data

Outperforms existing small models by over 15% in limited data settings

Abstract

Beam prediction is an effective approach to reduce training overhead in massive multiple-input multiple-output (MIMO) systems. However, existing beam prediction models still exhibit limited generalization ability in diverse scenarios, which remains a critical challenge. In this paper, we propose MLM-BP, a beam prediction framework based on the multi-modal large model released by DeepSeek, with full consideration of multi-modal environmental information. Specifically, the distribution of scatterers that impact the optimal beam is captured by the sensing devices. Then positions are tokenized to generate text-based representations, and multi-view images are processed by an image encoder, which is fine-tuned with low-rank adaptation (LoRA), to extract environmental embeddings. Finally, these embeddings are fed into the large model, and an output projection module is designed to determine the optimal beam index. Simulation results show that MLM-BP achieves 98.1% Top-1 accuracy on the simulation dataset. Additionally, it demonstrates few-shot generalization on a real-world dataset, achieving 72.7% Top-1 accuracy and 92.4% Top-3 accuracy with only 30% of the dataset, outperforming the existing small models by over 15%.