LWM-Temporal: Sparse Spatio-Temporal Attention for Wireless Channel Representation Learning

TL;DR

LWM-Temporal introduces a geometry-aware, sparse spatio-temporal attention model for wireless channel representation learning, enabling universal, transferable embeddings that improve prediction accuracy across mobility scenarios.

Contribution

It proposes a novel Sparse Spatio-Temporal Attention mechanism and a physics-informed self-supervised pretraining approach for universal wireless channel modeling.

Findings

Improves channel prediction accuracy across mobility regimes.

Reduces attention complexity by an order of magnitude.

Enhances transferability with geometry-aware pretraining.

Abstract

LWM-Temporal is a new member of the Large Wireless Models (LWM) family that targets the spatiotemporal nature of wireless channels. Designed as a task-agnostic foundation model, LWM-Temporal learns universal channel embeddings that capture mobility-induced evolution and are reusable across various downstream tasks. To achieve this objective, LWM-Temporal operates in the angle-delay-time domain and introduces Sparse Spatio-Temporal Attention (SSTA), a propagation-aligned attention mechanism that restricts interactions to physically plausible neighborhoods, reducing attention complexity by an order of magnitude while preserving geometry-consistent dependencies. LWM-Temporal is pretrained in a self-supervised manner using a physics-informed masking curriculum that emulates realistic occlusions, pilot sparsity, and measurement impairments. Experimental results on channel prediction across multiple mobility regimes show consistent improvements over strong baselines, particularly under long horizons and limited fine-tuning data, highlighting the importance of geometry-aware architectures and geometry-consistent pretraining for learning transferable spatiotemporal wireless representations.