Bandit-Based Rate Adaptation for a Single-Server Queue

TL;DR

This paper introduces a bandit-based algorithm for rate adaptation in a single-server queue with partial feedback, achieving near-optimal bounds on queue size depending on the knowledge of system slack.

Contribution

It proposes a phased discretization algorithm for unknown slack and establishes bounds showing near-optimal performance when slack is known.

Findings

Proposed a phased algorithm that refines rate discretization without knowing slack.

Achieved a bound of O(log^{3.5}(1/ε)/ε^3) on average queue size.

Proved a lower bound of Ω(1/ε^2) on worst-case queue size for any algorithm.

Abstract

This paper considers the problem of obtaining bounded time-average expected queue sizes in a single-queue system with a partial-feedback structure. Time is slotted; in slot $t$ the transmitter chooses a rate $V(t)$ from a continuous interval. Transmission succeeds if and only if $V(t)\le C(t)$, where channel capacities $\{C(t)\}$ and arrivals are i.i.d. draws from fixed but unknown distributions. The transmitter observes only binary acknowledgments (ACK/NACK) indicating success or failure. Let $\varepsilon>0$ denote a sufficiently small lower bound on the slack between the arrival rate and the capacity region. We propose a phased algorithm that progressively refines a discretization of the uncountable infinite rate space and, without knowledge of $\varepsilon$, achieves a $\mathcal{O}\!\big(\log^{3.5}(1/\varepsilon)/\varepsilon^{3}\big)$ time-average expected queue size uniformly over the horizon. We also prove a converse result showing that for any rate-selection algorithm, regardless of whether $\varepsilon$ is known, there exists an environment in which the worst-case time-average expected queue size is $\Omega(1/\varepsilon^{2})$. Thus, while a gap remains in the setting without knowledge of $\varepsilon$, we show that if $\varepsilon$ is known, a simple single-stage UCB type policy with a fixed discretization of the rate space achieves $\mathcal{O}\!\big(\log(1/\varepsilon)/\varepsilon^{2}\big)$, matching the converse up to logarithmic factors.