Safe RAN control: A Symbolic Reinforcement Learning Approach

TL;DR

This paper introduces a symbolic reinforcement learning architecture for safe control of RAN, enabling automated high-level safety specifications and ensuring network performance through model-checking techniques.

Contribution

It presents a novel SRL-based framework with a user interface for safety-aware RAN control, integrating model-checking with reinforcement learning for the first time.

Findings

Successfully enforces safety constraints in RAN control

Allows user-defined safety specifications via UI

Demonstrates effective safe RET optimization

Abstract

In this paper, we present a Symbolic Reinforcement Learning (SRL) based architecture for safety control of Radio Access Network (RAN) applications. In particular, we provide a purely automated procedure in which a user can specify high-level logical safety specifications for a given cellular network topology in order for the latter to execute optimal safe performance which is measured through certain Key Performance Indicators (KPIs). The network consists of a set of fixed Base Stations (BS) which are equipped with antennas, which one can control by adjusting their vertical tilt angle. The aforementioned process is called Remote Electrical Tilt (RET) optimization. Recent research has focused on performing this RET optimization by employing Reinforcement Learning (RL) strategies due to the fact that they have self-learning capabilities to adapt in uncertain environments. The term safety refers to particular constraints bounds of the network KPIs in order to guarantee that when the algorithms are deployed in a live network, the performance is maintained. In our proposed architecture the safety is ensured through model-checking techniques over combined discrete system models (automata) that are abstracted through the learning process. We introduce a user interface (UI) developed to help a user set intent specifications to the system, and inspect the difference in agent proposed actions, and those that are allowed and blocked according to the safety specification.