An Experimental Reservoir-Augmented Foundation Model: 6G O-RAN Case Study

TL;DR

This paper introduces RA-MAT, a reservoir-augmented transformer model designed for efficient, real-time analysis of high-dimensional, non-stationary time series data in 6G O-RAN networks, enabling scalable KPI forecasting.

Contribution

It proposes a novel reservoir-augmented masked autoencoding transformer that combines echo state networks with self-supervised learning for efficient 6G network analytics.

Findings

Achieved sub-0.06 MSE on KPI forecasting tasks.

Reduced computational complexity by replacing quadratic attention with linear operations.

Demonstrated effective adaptation to diverse downstream tasks.

Abstract

Next-generation open radio access networks (O-RAN) continuously stream tens of key performance indicators (KPIs) together with raw in-phase/quadrature (IQ) samples, yielding ultra-high-dimensional, non-stationary time series that overwhelm conventional transformer architectures. We introduce a reservoir-augmented masked autoencoding transformer (RA-MAT). This time series foundation model employs echo state network (ESN) computing with masked autoencoding to satisfy the stringent latency, energy efficiency, and scalability requirements of 6G O-RAN testing. A fixed, randomly initialized ESN rapidly projects each temporal patch into a rich dynamical embedding without backpropagation through time, converting the quadratic self-attention bottleneck into a lightweight linear operation. These embeddings drive a patch-wise masked autoencoder that reconstructs 30% randomly masked patches, compelling the encoder to capture both local dynamics and long-range structure from unlabeled data. After self-supervised pre-training, RA-MAT is fine-tuned with a shallow task head while keeping the reservoir and most transformer layers frozen, enabling low-footprint adaptation to diverse downstream tasks such as O-RAN KPI forecasting. In a comprehensive O-RAN KPI case study, RA-MAT achieved sub-0.06 mean squared error (MSE) on several continuous and discrete KPIs. This work positions RA-MAT as a practical pathway toward real-time, foundation-level analytics in future 6G networks.