Symbolic Reinforcement Learning for Safe RAN Control

TL;DR

This paper introduces a symbolic reinforcement learning architecture that ensures safe control in Radio Access Networks by integrating high-level safety specifications with model-checking techniques to shield RL agents.

Contribution

It presents a novel SRL architecture that combines LTL-based safety specifications, model-checking, and RL for safe RAN control, with a user interface for setting and inspecting safety constraints.

Findings

Effective safety shielding through model-checking in RAN control

User interface facilitates specification and inspection of safety constraints

Demonstrated integration of LTL specifications with RL for network safety

Abstract

In this paper, we demonstrate a Symbolic Reinforcement Learning (SRL) architecture for safe control in Radio Access Network (RAN) applications. In our automated tool, a user can select a high-level safety specifications expressed in Linear Temporal Logic (LTL) to shield an RL agent running in a given cellular network with aim of optimizing network performance, as measured through certain Key Performance Indicators (KPIs). In the proposed architecture, network safety shielding is ensured through model-checking techniques over combined discrete system models (automata) that are abstracted through reinforcement learning. We demonstrate the user interface (UI) helping the user set intent specifications to the architecture and inspect the difference in allowed and blocked actions.