Model Based Residual Policy Learning with Applications to Antenna Control

TL;DR

This paper introduces Model-Based Residual Policy Learning (MBRPL), a reinforcement learning approach tailored for antenna control that improves sample efficiency and initial performance in real network applications.

Contribution

It is the first to apply a model-based RL approach specifically to antenna control, enhancing existing policies with better sample efficiency and practical deployment potential.

Findings

MBRPL achieves strong initial performance.

It significantly improves sample efficiency over traditional RL methods.

The approach is effective for real-world antenna control applications.

Abstract

Non-differentiable controllers and rule-based policies are widely used for controlling real systems such as telecommunication networks and robots. Specifically, parameters of mobile network base station antennas can be dynamically configured by these policies to improve users coverage and quality of service. Motivated by the antenna tilt control problem, we introduce Model-Based Residual Policy Learning (MBRPL), a practical reinforcement learning (RL) method. MBRPL enhances existing policies through a model-based approach, leading to improved sample efficiency and a decreased number of interactions with the actual environment when compared to off-the-shelf RL methods.To the best of our knowledge, this is the first paper that examines a model-based approach for antenna control. Experimental results reveal that our method delivers strong initial performance while improving sample efficiency over previous RL methods, which is one step towards deploying these algorithms in real networks.