Practical Policy Distillation for Reinforcement Learning in Radio Access Networks

TL;DR

This paper explores policy distillation techniques to create compact, generalizable reinforcement learning models for radio access networks, addressing hardware constraints and maintaining performance across diverse network scenarios.

Contribution

It introduces single-policy and multi-policy distillation methods tailored for resource-limited RAN hardware, enabling deployment of effective RL models in legacy 4G and 5G systems.

Findings

Distilled models retain teacher performance in diverse scenarios

Both strategies produce models under 1MB with sub-100μs inference

Experimental results validate effectiveness in 5G-compliant simulators

Abstract

Adopting artificial intelligence (AI) in radio access networks (RANs) presents several challenges, including limited availability of link-level measurements (e.g., CQI reports), stringent real-time processing constraints (e.g., sub-1 ms per TTI), and network heterogeneity (different spectrum bands, cell types, and vendor equipment). A critical yet often overlooked barrier lies in the computational and memory limitations of RAN baseband hardware, particularly in legacy 4th Generation (4G) systems, which typically lack on-chip neural accelerators. As a result, only lightweight AI models (under 1 Mb and sub-100~\mu s inference time) can be effectively deployed, limiting both their performance and applicability. However, achieving strong generalization across diverse network conditions often requires large-scale models with substantial resource demands. To address this trade-off, this paper investigates policy distillation in the context of a reinforcement learning-based link adaptation task. We explore two strategies: single-policy distillation, where a scenario-agnostic teacher model is compressed into one generalized student model; and multi-policy distillation, where multiple scenario-specific teachers are consolidated into a single generalist student. Experimental evaluations in a high-fidelity, 5th Generation (5G)-compliant simulator demonstrate that both strategies produce compact student models that preserve the teachers' generalization capabilities while complying with the computational and memory limitations of existing RAN hardware.