Distributive Stochastic Learning for Delay-Optimal OFDMA Power and Subband Allocation

TL;DR

This paper introduces a distributive stochastic learning algorithm for delay-optimal power and subband allocation in OFDMA uplink systems, effectively balancing complexity and performance through an auction-based approach.

Contribution

It develops a novel online learning framework that approximates the Q-factor distributively, enabling delay-optimal resource allocation with low complexity and guaranteed convergence.

Findings

The algorithm converges almost surely under the auction mechanism.

Delay-optimal power control exhibits a multi-level water-filling structure.

Computational complexity is linear in the number of users and subbands.

Abstract

In this paper, we consider the distributive queue-aware power and subband allocation design for a delay-optimal OFDMA uplink system with one base station, $K$ users and $N_F$ independent subbands. Each mobile has an uplink queue with heterogeneous packet arrivals and delay requirements. We model the problem as an infinite horizon average reward Markov Decision Problem (MDP) where the control actions are functions of the instantaneous Channel State Information (CSI) as well as the joint Queue State Information (QSI). To address the distributive requirement and the issue of exponential memory requirement and computational complexity, we approximate the subband allocation Q-factor by the sum of the per-user subband allocation Q-factor and derive a distributive online stochastic learning algorithm to estimate the per-user Q-factor and the Lagrange multipliers (LM) simultaneously and determine the control actions using an auction mechanism. We show that under the proposed auction mechanism, the distributive online learning converges almost surely (with probability 1). For illustration, we apply the proposed distributive stochastic learning framework to an application example with exponential packet size distribution. We show that the delay-optimal power control has the {\em multi-level water-filling} structure where the CSI determines the instantaneous power allocation and the QSI determines the water-level. The proposed algorithm has linear signaling overhead and computational complexity $\mathcal O(KN)$, which is desirable from an implementation perspective.