Risk-Sensitive Optimal Control of Queues

TL;DR

This paper analyzes risk-sensitive optimal control policies for queue management in wireless networks, demonstrating that threshold-based policies are optimal and how their thresholds depend on transmission costs.

Contribution

It establishes the optimality of threshold policies for risk-sensitive queue control and characterizes how thresholds vary with transmission costs.

Findings

Optimal control is threshold-based, depending on queue length.

Threshold increases with higher transmission costs.

Threshold policy is optimal within specific cost intervals.

Abstract

We consider the problem of designing risk-sensitive optimal control policies for scheduling packet transmissions in a stochastic wireless network. A single client is connected to an access point (AP) through a wireless channel. Packet transmission incurs a cost $C$, while packet delivery yields a reward of $R$ units. The client maintains a finite buffer of size $B$, and a penalty of $L$ units is imposed upon packet loss which occurs due to finite queueing buffer. We show that the risk-sensitive optimal control policy for such a simple set-up is of threshold type, i.e., it is optimal to carry out packet transmissions only when $Q(t)$, i.e., the queue length at time $t$ exceeds a certain threshold $\tau$. It is also shown that the value of threshold $\tau$ increases upon increasing the cost per unit packet transmission $C$. Furthermore, it is also shown that a threshold policy with threshold equal to $\tau$ is optimal for a set of problems in which cost $C$ lies within an interval $[C_l,C_u]$. Equations that need to be solved in order to obtain $C_l,C_u$ are also provided.