Realtime Spectrum Monitoring via Reinforcement Learning -- A Comparison Between Q-Learning and Heuristic Methods

TL;DR

This paper compares reinforcement learning and heuristic methods for real-time spectrum monitoring, demonstrating that Q-learning achieves higher detection rates in a simplified scenario, with tunable trade-offs between detection and exploration.

Contribution

It introduces a comparative analysis of Q-learning and heuristic approaches for spectrum resource management in real-time monitoring.

Findings

Q-learning outperforms heuristic methods in detection rate

Q-learning's performance can be tuned via parameters

Reinforcement learning offers a promising approach for spectrum management

Abstract

Due to technological advances in the field of radio technology and its availability, the number of interference signals in the radio spectrum is continuously increasing. Interference signals must be detected in a timely fashion, in order to maintain standards and keep emergency frequencies open. To this end, specialized (multi-channel) receivers are used for spectrum monitoring. In this paper, the performances of two different approaches for controlling the available receiver resources are compared. The methods used for resource management (ReMa) are linear frequency tuning as a heuristic approach and a Q-learning algorithm from the field of reinforcement learning. To test the methods to be investigated, a simplified scenario was designed with two receiver channels monitoring ten non-overlapping frequency bands with non-uniform signal activity. For this setting, it is shown that the Q-learning algorithm used has a significantly higher detection rate than the heuristic approach at the expense of a smaller exploration rate. In particular, the Q-learning approach can be parameterized to allow for a suitable trade-off between detection and exploration rate.