On Monotonicity of the Optimal Transmission Policy in Cross-layer Adaptive m-QAM Modulation

TL;DR

This paper investigates the monotonicity properties of the optimal transmission policy in a cross-layer adaptive m-QAM system modeled as an MDP, leading to more efficient algorithms for policy computation.

Contribution

It proves the monotonicity of the optimal policy using L-natural-convexity and submodularity, and introduces low-complexity algorithms leveraging these properties.

Findings

MPI based on L-natural-convexity significantly reduces computation time.

DSPSA effectively tracks the optimal policy under changing system parameters.

Monotonicity properties enable more efficient dynamic programming solutions.

Abstract

This paper considers a cross-layer adaptive modulation system that is modeled as a Markov decision process (MDP). We study how to utilize the monotonicity of the optimal transmission policy to relieve the computational complexity of dynamic programming (DP). In this system, a scheduler controls the bit rate of the m-quadrature amplitude modulation (m-QAM) in order to minimize the long-term losses incurred by the queue overflow in the data link layer and the transmission power consumption in the physical layer. The work is done in two steps. Firstly, we observe the L-natural-convexity and submodularity of DP to prove that the optimal policy is always nondecreasing in queue occupancy/state and derive the sufficient condition for it to be nondecreasing in both queue and channel states. We also show that, due to the L-natural-convexity of DP, the variation of the optimal policy in queue state is restricted by a bounded marginal effect: The increment of the optimal policy between adjacent queue states is no greater than one. Secondly, we use the monotonicity results to present two low complexity algorithms: monotonic policy iteration (MPI) based on L-natural-convexity and discrete simultaneous perturbation stochastic approximation (DSPSA). We run experiments to show that the time complexity of MPI based on L-natural-convexity is much lower than that of DP and the conventional MPI that is based on submodularity and DSPSA is able to adaptively track the optimal policy when the system parameters change.