Adaptive Requesting in Decentralized Edge Networks via Non-Stationary Bandits

TL;DR

This paper introduces a novel decentralized bandit algorithm for optimizing information freshness in edge networks, effectively handling non-stationary reward dynamics and partial observability.

Contribution

It proposes the AGING BANDIT WITH ADAPTIVE RESET algorithm, combining adaptive windowing and periodic monitoring to address non-stationary, coupled reward processes in decentralized settings.

Findings

The algorithm achieves near-optimal performance in non-stationary environments.

Theoretical guarantees are established for the proposed method.

Simulations validate the effectiveness of the approach.

Abstract

We study a decentralized collaborative requesting problem that aims to optimize the information freshness of time-sensitive clients in edge networks consisting of multiple clients, access nodes (ANs), and servers. Clients request content through ANs acting as gateways, without observing AN states or the actions of other clients. We define the reward as the age of information reduction resulting from a client's selection of an AN, and formulate the problem as a non-stationary multi-armed bandit. In this decentralized and partially observable setting, the resulting reward process is history-dependent and coupled across clients, and exhibits both abrupt and gradual changes in expected rewards, rendering classical bandit-based approaches ineffective. To address these challenges, we propose the AGING BANDIT WITH ADAPTIVE RESET algorithm, which combines adaptive windowing with periodic monitoring to track evolving reward distributions. We establish theoretical performance guarantees showing that the proposed algorithm achieves near-optimal performance, and we validate the theoretical results through simulations.