Throughput Optimization in UAV-Mounted RIS under Jittering and Imperfect CSI via DRL

TL;DR

This paper develops a deep reinforcement learning approach to optimize throughput in UAV-mounted RIS systems, effectively handling jitter and imperfect CSI, and significantly outperforming traditional methods in speed and robustness.

Contribution

It introduces a model-free DRL framework with a feasibility layer for UAV RIS throughput optimization under practical impairments, a novel approach in this context.

Findings

DRL algorithms outperform AO-WMMSE in throughput under severe jitter.

Proposed methods achieve near-benchmark performance with much faster inference times.

The framework effectively manages stochastic UAV jitter and imperfect CSI.

Abstract

Reconfigurable intelligent surfaces (RISs) mounted on unmanned aerial vehicles (UAVs) can reshape wireless propagation on-demand. However, their performance is sensitive to UAV jitter and cascaded channel uncertainty. This paper investigates a downlink multiple-input single-output UAV-mounted RIS system in which a ground multiple-antenna base station (BS) serves multiple single-antenna users under practical impairments. Our goal is to maximize the expected throughput under stochastic three-dimensional UAV jitter and imperfect cascaded channel state information (CSI) based only on the available channel estimates. This leads to a stochastic nonconvex optimization problem subject to a BS transmit power constraint and strict unit-modulus constraints on all RIS elements. To address this problem, we design a model-free deep reinforcement learning (DRL) framework with a contextual bandit formulation. A differentiable feasibility layer is utilized to map continuous actions to feasible solutions, while the reward is a Monte Carlo estimate of the expected throughput. We instantiate this framework with constrained variants of deep deterministic policy gradient (DDPG) and twin delayed deep deterministic policy gradient (TD3) that do not use target networks. Simulations show that the proposed algorithms yield higher throughput than conventional alternating optimization-based weighted minimum mean-square error (AO-WMMSE) baselines under severe jitter and low CSI quality. Across different scenarios, the proposed methods achieve performance that is either comparable to or slightly below the AO-WMMSE benchmark, based on sample average approximation (SAA) with a relative gap ranging from 0-12%. Moreover, the proposed DRL controllers achieve online inference times of 0.6 ms per decision versus roughly 370-550 ms for AO-WMMSE solvers.