Intelligent Resource Allocations for IRS-Assisted OFDM Communications: A Hybrid MDQN-DDPG Approach

TL;DR

This paper introduces a hybrid MDQN-DDPG algorithm to optimize resource allocation in IRS-assisted OFDM systems, effectively handling hybrid action spaces to maximize system sum rate.

Contribution

It proposes a novel hybrid deep reinforcement learning approach combining MDQN and DDPG for joint optimization in IRS-assisted OFDM communications.

Findings

Outperforms traditional schemes in sum rate.

Effectively manages hybrid discrete-continuous actions.

Demonstrates improved learning and adaptation.

Abstract

In this paper, we study the resource allocation problem for an intelligent reflecting surface (IRS)-assisted OFDM system. The system sum rate maximization framework is formulated by jointly optimizing subcarrier allocation, base station transmit beamforming and IRS phase shift. Considering the continuous and discrete hybrid action space characteristics of the optimization variables, we propose an efficient resource allocation algorithm combining multiple deep Q networks (MDQN) and deep deterministic policy-gradient (DDPG) to deal with this issue. In our algorithm, MDQN are employed to solve the problem of large discrete action space, while DDPG is introduced to tackle the continuous action allocation. Compared with the traditional approaches, our proposed MDQN-DDPG based algorithm has the advantage of continuous behavior improvement through learning from the environment. Simulation results demonstrate superior performance of our design in terms of system sum rate compared with the benchmark schemes.