# Relay Selection in Wireless Networks as Restless Bandits

## TL;DR

This paper models relay selection in wireless networks as a restless bandit problem, proves indexability, and proposes a Whittle index-based policy that improves cost, delay, and throughput over previous methods.

## Contribution

It demonstrates that relay selection can be formulated as a Whittle indexable restless bandit problem and develops an efficient index computation method.

## Key findings

- Proposed policy outperforms prior relay selection methods.
- The relay selection problem is proven to be Whittle-indexable.
- Simulation results show improvements in cost, delay, and throughput.

## Abstract

We consider a wireless network in which a source node needs to transmit a large file to a destination node. The direct wireless link between the source and the destination is assumed to be blocked. Multiple candidate relays are available to forward packets from the source to the destination. A holding cost is incurred for each packet stored at every relay in each time slot. The objective is to design a policy for selecting a relay in each time slot to which the source attempts to send a packet, so as to minimize the expected long-run time-averaged total packet holding cost at the relays. This problem is an instance of the restless multi-armed bandit (RMAB) problem, which is provably hard to solve. We prove that this relay selection problem is Whittle-indexable, and propose a method to compute the Whittle index of each relay in every time slot. In each time slot, our relay selection policy transmits a packet to the relay with the smallest Whittle index. Using simulations, we show that the proposed policy outperforms the relay selection policies proposed in prior work in terms of average cost, delay, as well as throughput.