Actor-Critic Deep Reinforcement Learning for Dynamic Multichannel Access

TL;DR

This paper introduces a model-free actor-critic deep reinforcement learning framework to optimize dynamic multichannel access in POMDP settings, demonstrating adaptability and improved performance over DQN-based methods.

Contribution

It presents a novel actor-critic RL approach for multichannel access, capable of handling uncertainty and environment changes, outperforming existing DQN methods.

Findings

The framework achieves higher average rewards than DQN.

It demonstrates robustness against different channel switching patterns.

The method adapts effectively to time-varying environments.

Abstract

We consider the dynamic multichannel access problem, which can be formulated as a partially observable Markov decision process (POMDP). We first propose a model-free actor-critic deep reinforcement learning based framework to explore the sensing policy. To evaluate the performance of the proposed sensing policy and the framework's tolerance against uncertainty, we test the framework in scenarios with different channel switching patterns and consider different switching probabilities. Then, we consider a time-varying environment to identify the adaptive ability of the proposed framework. Additionally, we provide comparisons with the Deep-Q network (DQN) based framework proposed in [1], in terms of both average reward and the time efficiency.