Decentralized Age-of-Information Bandits

TL;DR

This paper addresses the challenge of scheduling multiple data sources over multiple channels to minimize Age-of-Information (AoI) using multi-armed bandit algorithms, proposing new policies with performance guarantees.

Contribution

It introduces novel AoI-aware policies based on UCB and Thompson Sampling for distributed multi-armed bandit problems with unknown channel statistics.

Findings

Proven performance guarantees for UCB-based policy

Development of a Thompson Sampling-based policy

Simulation results showing improved AoI performance

Abstract

Age-of-Information (AoI) is a performance metric for scheduling systems that measures the freshness of the data available at the intended destination. AoI is formally defined as the time elapsed since the destination received the recent most update from the source. We consider the problem of scheduling to minimize the cumulative AoI in a multi-source multi-channel setting. Our focus is on the setting where channel statistics are unknown and we model the problem as a distributed multi-armed bandit problem. For an appropriately defined AoI regret metric, we provide analytical performance guarantees of an existing UCB-based policy for the distributed multi-armed bandit problem. In addition, we propose a novel policy based on Thomson Sampling and a hybrid policy that tries to balance the trade-off between the aforementioned policies. Further, we develop AoI-aware variants of these policies in which each source takes its current AoI into account while making decisions. We compare the performance of various policies via simulations.